-- Logs begin at Sat 2022-07-16 11:59:19 UTC, end at Sat 2022-07-16 13:17:48 UTC. -- Jul 16 12:28:54.045964 dv-cinder-836973-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 16 12:28:54 2022] *** Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: nodename: dv-cinder-836973-12-iscsi Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: machine: x86_64 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: clock source: unix Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: pcre jit disabled Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: detected number of CPU cores: 4 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: current working directory: / Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: detected binary path: /usr/bin/uwsgi-core Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: your processes number limit is 47680 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: your memory page size is 4096 bytes Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: detected max file descriptor number: 2048 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: lock engine: pthread robust mutexes Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: thunder lock: enabled Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 16 12:28:54.060692 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 16 12:28:54.097955 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: Python main interpreter initialized at 0x55ebe3eec9c0 Jul 16 12:28:54.097955 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: python threads support enabled Jul 16 12:28:54.097955 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: your server socket listen backlog is limited to 100 connections Jul 16 12:28:54.097955 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: your mercy for graceful operations on workers is 90 seconds Jul 16 12:28:54.098570 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: mapped 403077 bytes (393 KB) for 2 cores Jul 16 12:28:54.098656 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: *** Operational MODE: preforking *** Jul 16 12:28:54.098899 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: *** uWSGI is running in multiple interpreter mode *** Jul 16 12:28:54.098899 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: spawned uWSGI master process (pid: 80426) Jul 16 12:28:54.099103 dv-cinder-836973-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 16 12:28:54.100014 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: spawned uWSGI worker 1 (pid: 80440, cores: 1) Jul 16 12:28:54.101807 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: spawned uWSGI worker 2 (pid: 80441, cores: 1) Jul 16 12:28:54.101807 dv-cinder-836973-12-iscsi devstack@keystone.service[80426]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 16 12:28:59.587874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 12:28:59.587874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: from cryptography.utils import int_from_bytes Jul 16 12:28:59.588628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 12:28:59.588628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jul 16 12:28:59.588628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 12:28:59.588628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jul 16 12:28:59.589187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 12:28:59.589187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: from cryptography.utils import int_from_bytes Jul 16 12:28:59.637255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 12:28:59.637419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 12:28:59.637501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 12:28:59.637574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 12:28:59.637641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] command line args: [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 12:28:59.637706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] command line args: [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 12:28:59.637886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 12:28:59.637981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 12:28:59.638055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ================================================================================ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 12:28:59.638125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ================================================================================ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 12:28:59.638272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] admin_token = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_dir = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_dir = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_source = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.638943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_source = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] debug = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] debug = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.639903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_config_append = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_config_append = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.640937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_dir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_dir = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.641989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.642957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.643689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.643788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.643912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.644952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.645821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] publish_errors = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] publish_errors = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.646906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] standard_threads = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] standard_threads = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_journal = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.647947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_json = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_stderr = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_syslog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_journal = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_json = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.648852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.648927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_stderr = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.649045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_syslog = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.649125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.649292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 12:28:59.649369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.649549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.649627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.649787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.649872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.650984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.651926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.652970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.653952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.654926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.655995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.656957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.657928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.658991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.659869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.660934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.661969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.662892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.663975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.664982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.665899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.666908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.667999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.668931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.669921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.670926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.671991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.672936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.external = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.673978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.external = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.password = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.674910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.token = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.password = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.token = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.675851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.676975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.677954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.678989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.679915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.680953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.681923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.682889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.683996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.684949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.685875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.686964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.687941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.688977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.689716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.690978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.691934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.692957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.693909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.694858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.695953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.696975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.697945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.698868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.699959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.700990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.701943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.702887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.703963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.704972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.705151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.705233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.705386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.705467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.706827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.707961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.708988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.709981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.710939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.711899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.712938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.713935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.714962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.715913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.716995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.717975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.718930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.719950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.720901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.721887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.722857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.723978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.724923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.725827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 12:28:59.725907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.726117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.726360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.726605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 12:28:59.726779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 12:28:59.757413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.757584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.757685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.757766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.757842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.757920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.766147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.766251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.794433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.794668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.795689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.795921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.937459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.938328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.988028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.988164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.988266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:28:59.988374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.053958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.054097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.054182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.054333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.054558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.054777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.079654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.079883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.079974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.080897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.081183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.081183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.081431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.081431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.092240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.092588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.105938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.106876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.107661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 12:29:00.124998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.125238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 12:29:00.125525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.125653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.125785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 12:29:00.125928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 12:29:00.126060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.126191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.126332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 12:29:00.126462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.126611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.126746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.126874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.127004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.127133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.127264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.137943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.138224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.138494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.138645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.138785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.138914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.139166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.139442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.146978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.148032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.150303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.150588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.150982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.151269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.151407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.151541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.151680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.151811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.151942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.152899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.153029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.153197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.153329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.153533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.153801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.154072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.180886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.182128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.182612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.182901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.183758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.184035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.188261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.188503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.188788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.189298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.189572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.189878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.196655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.196941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.197628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.197908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.201626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.201917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.202202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.202484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.202566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.202759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.202844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.203844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.204108 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.204369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.223315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.223599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.224040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.224312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.228190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.228808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.231929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.232226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.232447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.232738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.237308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.237907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.240952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.241539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.244735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.245940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.246137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.246229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.246418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.246693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.265837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.266222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.266445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.266551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.266756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.266840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.267080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.267347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.275990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.276391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.277780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.278059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.278142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.278410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.281133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.281421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.281503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.281715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.281799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.281998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.282082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.282281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.282365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.282567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.295957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.296063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.296239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.296324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.296510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.296592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.296785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.296870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.297927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.317776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.317875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.321009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.321116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.324334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.324426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.325944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.326029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.327581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.327674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.327860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.327944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.328140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.328231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.336197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.336341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.337837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.338000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.341094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.341365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.341450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.341656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.341739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.341930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.342015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.342208 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.351468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.351750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.351935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.352021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.352224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.352506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.359820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.360096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.360371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.360455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 12:29:00.360637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.360720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.360915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.361888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.362889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.363911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.364181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 12:29:00.402291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ebe3eec9c0 pid: 80440 (default app) Jul 16 12:29:00.403485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ebe3eec9c0 pid: 80441 (default app) Jul 16 12:29:00.405698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02864f-10c8-499d-8637-ddc9927b0780 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:00.405992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02864f-10c8-499d-8637-ddc9927b0780 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:00.406269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02864f-10c8-499d-8637-ddc9927b0780 None None] PATH_INFO: `/v3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:00.410744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Sat Jul 16 12:29:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:08.794117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:08.794690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:08.794690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:08.798987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:08.804752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:08.807410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:08.807410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:08.875022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: INFO dbcounter [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] Registered counter for database keystone Jul 16 12:29:08.896854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG oslo_db.sqlalchemy.engines [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80440) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 12:29:08.902995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writer thread running {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 12:29:09.081110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80440) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 16 12:29:09.081502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80440) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 16 12:29:09.136464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:09.138121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:09.260905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 16 12:29:09.416950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:09.487752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: INFO dbcounter [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] Registered counter for database keystone Jul 16 12:29:09.506809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG oslo_db.sqlalchemy.engines [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 12:29:09.513056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writer thread running {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 12:29:09.634730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a19e1-ed26-4ae6-8966-017a375f8183 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:09.639007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:09.639665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:09.639954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:09.640231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:09.648965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:09.700675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING keystone.common.rbac_enforcer.enforcer [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 16 12:29:09.700964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0458b0-eea7-4f80-aafd-e76370b6dfee admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:09.711507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Sat Jul 16 12:29:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:29:09.717578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:09.732897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c8606-6fdd-41a7-9c9f-08c6441d2b5c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:09.737366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:09.738499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:09.738794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:09.739074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] PATH_INFO: `/v3/projects/5d986eebab5a45d29f4ae51c867a19af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:09.743162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] RBAC: Authorizing `identity:get_project(project_id=5d986eebab5a45d29f4ae51c867a19af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:09.794120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING keystone.common.rbac_enforcer.enforcer [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 16 12:29:09.794463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41372bef-adab-40c6-aa3a-3c5a15029ad5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:09.798972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Sat Jul 16 12:29:09 2022] GET /identity/v3/projects/5d986eebab5a45d29f4ae51c867a19af => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:11.790548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6cdf-ae64-4027-9cb0-c404b77777f7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:11.791087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6cdf-ae64-4027-9cb0-c404b77777f7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:11.791087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6cdf-ae64-4027-9cb0-c404b77777f7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:11.795298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:11.801064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:11.801534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:11.801626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:11.819911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:11.821455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:11.829305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 16 12:29:11.852163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6cdf-ae64-4027-9cb0-c404b77777f7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:11.864282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6cdf-ae64-4027-9cb0-c404b77777f7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:11.868463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:11.869121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:11.869409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:11.869687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:11.877604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:11.879512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607083-07b4-406a-a3c8-286a87a98075 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:11.888183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Sat Jul 16 12:29:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:29:11.894430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:11.903322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1222d-30a1-4c92-ba5f-7ca5904ef369 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:11.907704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:11.908552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:11.908835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:11.909144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:11.911134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:11.912945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bdcc-ed55-43c4-87a1-0cbe6b0f09f6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:11.914539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Sat Jul 16 12:29:11 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:13.986945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4d72e-0205-4881-890a-bca0340dd11b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:13.988027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4d72e-0205-4881-890a-bca0340dd11b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:13.988170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4d72e-0205-4881-890a-bca0340dd11b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:13.989505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:13.994588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:13.995025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:13.995149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.012085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:14.013295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:14.015391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc25d5-d7b8-4072-823a-2eedc053b66b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.015683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc25d5-d7b8-4072-823a-2eedc053b66b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.015966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc25d5-d7b8-4072-823a-2eedc053b66b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.017339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:14.022713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.023006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.023286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.063185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG passlib.handlers.bcrypt [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80441) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 16 12:29:14.063600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG passlib.handlers.bcrypt [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80441) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 16 12:29:14.093596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:14.109222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:14.110795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:14.116855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.140505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.216426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:14.221640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b373489-81e4-4687-88ed-a803a5c54668 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.238774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.256069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.256867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.257189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.257469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.268705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:14.279855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:14.280819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:14.281760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:14.288897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.339070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62694-ff18-46d6-af62-f27fa923db52 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.341358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.369410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.375608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.376514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.376802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.377111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.386437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a8a62-6652-4723-bf30-a4165d26f582 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.420480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.421164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.421445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.421724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.422475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:14.422777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.424155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.436358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:14.442117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.443793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.445327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.542275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158de18a-e7ae-415a-9009-b2e9e2b27061 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.543399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:29:14.549002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.552695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.575569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.578441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.578441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.578441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.578441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.579630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.581366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:14.586809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.601945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.647344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d32d6-8f0b-46a4-91a7-8f1e6b3c5e09 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.680712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.681745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.682026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.682300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.686344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93e55d-a65f-4ec9-a336-ca2213811dae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.700821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.702436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.719366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.720148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.720432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.720708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.722751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.724187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.725827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:29:14.726681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:14.727622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:14.728436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:29:14.728436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:29:14.728436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:29:14.728436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:14.728436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:14.731473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.733304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:14.789699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.811204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:14.830188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12ba79-d080-4512-b6ee-c492e5db10cb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.832672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98776ec3-714e-420e-994c-2fc06d45e6b8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:14.864186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.864838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.865155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.865431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.866259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.domain.id=default, user.id=29ed4c3c304b4678a24977fbdcf04135) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:14.866450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:14.866569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.867117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:14.867404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:14.867684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:14.867980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac95a6-fe2a-4684-8b43-04891c84f7f7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.868416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:14.868728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:14.870139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e9c4e-2a03-4894-9abf-5cac1dda03d5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:14.874646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/role_assignments?user.id=29ed4c3c304b4678a24977fbdcf04135&scope.domain.id=default&role.id=a495d38f67e447309647ac782785bd87 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:14.889826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:17.435595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7907-af70-4834-a236-1ee015bc834f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.436601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7907-af70-4834-a236-1ee015bc834f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.437120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7907-af70-4834-a236-1ee015bc834f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.438712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:17.444317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.444744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.444913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.448642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acfb85-0dd0-4aeb-9169-c2e55fd2b8b2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.449194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acfb85-0dd0-4aeb-9169-c2e55fd2b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.449676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acfb85-0dd0-4aeb-9169-c2e55fd2b8b2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.451173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 13/23] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:17.457650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.458223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.458714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.470390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:17.472372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:17.485983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:17.488219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:17.610964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 11/24] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:17.631361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 14/25] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:17.645500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:17.664499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:17.727534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:17.785749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:17.821389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0692a-0922-4b35-bd66-585b28dc7677 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:17.876972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b4eb9-12a4-4b82-b71f-f875b49f1361 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:17.904716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:17.905560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.905845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.906121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.926451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e736a110-80bd-4c62-ad9a-05fa5e36e39a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:17.934129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:17.934974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:17.935719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 15/26] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:17.939478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e533-670d-4a66-b3b3-dd59c9f75401 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.939768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e533-670d-4a66-b3b3-dd59c9f75401 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.940880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e533-670d-4a66-b3b3-dd59c9f75401 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.942412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 16/27] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:17.943666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af403-0d68-4a36-822a-201d0d4b3f22 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.944189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af403-0d68-4a36-822a-201d0d4b3f22 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.944604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af403-0d68-4a36-822a-201d0d4b3f22 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.945988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 17/28] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:17.950379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752af403-0d68-4a36-822a-201d0d4b3f22 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:17.952897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:17.953741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.954031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.954308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:17.965139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:17.966832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4b99faf8-54f4-4516-bf1d-63a147e08a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:29:17.985174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:17.985974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:17.986786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:29:17.986786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:29:17.986786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:29:17.986786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:17.989620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 12/29] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:17.991029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01030ca-7111-478d-a2ad-0f085d4f46a3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:17.991321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01030ca-7111-478d-a2ad-0f085d4f46a3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:17.991610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01030ca-7111-478d-a2ad-0f085d4f46a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.008372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f01030ca-7111-478d-a2ad-0f085d4f46a3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:18.009767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f01030ca-7111-478d-a2ad-0f085d4f46a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:18.067219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752af403-0d68-4a36-822a-201d0d4b3f22 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.092649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 13/30] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:18.094311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.094810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.095275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.113161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:18.114597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:18.118796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.119533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.119827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.120107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.120871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:18.121223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.122612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.124374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:18.132808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 18/31] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:18.137927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.142493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:18.194484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 14/32] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:18.195636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.270052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1c8a-0597-4350-909b-bb0d0b9f816c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.307629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.308529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.308980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd25510-0e9f-4995-a3d6-8e69bdd10178 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.309143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.309290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.310032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:18.310339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.311844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.331472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 19/33] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:18.332389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.348890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.349966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.350397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.350816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.351805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.353402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.398132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:18.442743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9523ab-aa0d-47f1-a94e-cf20078d6f9d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.485565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.486496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.486840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.487652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.489016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.491516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.494244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 20/34] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:18 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:18.495064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.589155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 15/35] 10.222.0.47 () {60 vars in 1131 bytes} [Sat Jul 16 12:29:18 2022] POST /identity/v3/roles => generated 219 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:18.590173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.613066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:18.615709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff6585-3806-48ff-90f8-b6d46032760c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.650528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:18.675092 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.678309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.680134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.680566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.681575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.684492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f41dfd-94f5-4a36-9b6a-296835485216 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.730290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0d9cf-091a-4f30-b706-2ed99705adf4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:18.768791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:18.769504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.769791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.770065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.771058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:18.772650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:18.869829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 16/36] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:29:18 2022] POST /identity/v3/projects => generated 301 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:18.876332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:18.891118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 21/37] 10.222.0.47 () {60 vars in 1131 bytes} [Sat Jul 16 12:29:18 2022] POST /identity/v3/roles => generated 225 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:18.892745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f46a9-ee61-4d5a-a344-8bc13b07bc73 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.893062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f46a9-ee61-4d5a-a344-8bc13b07bc73 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.893506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f46a9-ee61-4d5a-a344-8bc13b07bc73 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.894913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 22/38] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:18.896186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b2a9-bd60-474d-8326-0ed1e2800eb9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.896480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b2a9-bd60-474d-8326-0ed1e2800eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.896765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b2a9-bd60-474d-8326-0ed1e2800eb9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.898176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 23/39] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:18.900703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8187-9814-4d50-9743-5dccc6560e19 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:18.901110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8187-9814-4d50-9743-5dccc6560e19 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:18.901466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8187-9814-4d50-9743-5dccc6560e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:18.922677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-aeed8187-9814-4d50-9743-5dccc6560e19 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:18.924589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aeed8187-9814-4d50-9743-5dccc6560e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:18.974585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:19.018527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6971f-7736-40d5-9603-27770da48904 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:19.031196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 24/40] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:19.032689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.032992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.033322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.066037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:19.067303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.067802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.068285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.070066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:19.071982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:19.074312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 17/41] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:19.075444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:19.177446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:19.179050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:19.198407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec026a-49da-4a31-93cd-3c7fcca57e2f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:19.232932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:19.233640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.233925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.234203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.234945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:19.236325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:19.247625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 25/42] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:19.248442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:19.293571 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:19.318922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:19.324811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 18/43] 10.222.0.47 () {60 vars in 1131 bytes} [Sat Jul 16 12:29:19 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:19.325644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:19.351051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf7106-a184-478b-9d49-0b46d26809da None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:19.391731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:19.392627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.392910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.393773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] PATH_INFO: `/v3/domains/default/users/29ed4c3c304b4678a24977fbdcf04135/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.396050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=29ed4c3c304b4678a24977fbdcf04135, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:19.397783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:19.438531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc662d-5ac0-4f1e-8c2a-37b53062479e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:19.462486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 26/44] 10.222.0.47 () {58 vars in 1361 bytes} [Sat Jul 16 12:29:19 2022] PUT /identity/v3/domains/default/users/29ed4c3c304b4678a24977fbdcf04135/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:19.480065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:19.480909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.481752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.481752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.482906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:19.484399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f75d-0084-429b-913e-39c4efc10956 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:19.486894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 19/45] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:19.492174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:19.614142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ebefa-af74-470d-99b5-1c8cd6b5a594 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:19.618100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:19.651332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:19.651996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:19.652283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:19.652564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:19.653322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:19.654763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad480d9c-197a-4dee-a45b-3a2e4fab68b7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:19.742301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 27/46] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:29:19 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:21.476696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed2dc-751d-4374-b1d2-afc1547007d3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.477225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed2dc-751d-4374-b1d2-afc1547007d3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.477225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed2dc-751d-4374-b1d2-afc1547007d3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.478707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 20/47] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:21.483619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b906e-251f-4cee-9801-5f37bbdbbbda None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.484011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b906e-251f-4cee-9801-5f37bbdbbbda None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.484296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b906e-251f-4cee-9801-5f37bbdbbbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.501348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-444b906e-251f-4cee-9801-5f37bbdbbbda None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:21.503313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-444b906e-251f-4cee-9801-5f37bbdbbbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:21.569586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 28/48] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:21.592618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed2dc-751d-4374-b1d2-afc1547007d3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:21.659954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:21.694180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed2dc-751d-4374-b1d2-afc1547007d3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:21.728063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:21.728788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.729106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.729389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.741219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:21.746148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d33b6-ad99-4199-a3eb-ccaaaa316cf8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.746446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d33b6-ad99-4199-a3eb-ccaaaa316cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.746729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d33b6-ad99-4199-a3eb-ccaaaa316cf8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.748111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 29/49] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:21.749956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:21.750796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:21.751943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 21/50] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:21.753111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.753267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.753548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.756706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:21.771309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:21.772513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:21.839924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 30/51] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:21.856926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b2cf7-6e69-4cb5-90f0-ecdcebd92bfd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:21.863336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:21.881092 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:21.882814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:21.891017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:21.891681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.891960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.894375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.894375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:21.894375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:21.895461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:21.907012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 22/52] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:21.913152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:21.961552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a81b-87b8-490b-8533-bbc4bb8c9234 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:21.997202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:21.997928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:21.998216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:21.998499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:21.999678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:22.001191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:22.002869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 31/53] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:22.008291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:22.011238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5958-3666-43d1-a61c-3ee45a50fceb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:22.046420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:22.047296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:22.047583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:22.047863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:22.049868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:22.051318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:22.052945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 23/54] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:21 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:22.058253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:22.094581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:22.112677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694fc6-b557-4eeb-9fb4-24945f068750 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:22.148899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:22.149608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:22.149891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:22.150166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:22.150915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:22.152389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:22.219931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:22.261892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3b17-f554-4053-b2eb-0feef2fca0c8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:22.269784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 32/55] 10.222.0.47 () {60 vars in 1140 bytes} [Sat Jul 16 12:29:22 2022] POST /identity/v3/projects => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:22.300001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:22.300878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:22.301201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:22.301481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:22.302730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:22.304216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97918d4-d384-43a4-9677-a408db1388ad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:22.305976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 24/56] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:22.312158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:22.420551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb151d-45c5-416e-a7c8-a5e9e7e1bc10 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:22.455931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:22.456775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:22.457087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:22.457371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:22.458216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.domain.id=default, user.id=29ed4c3c304b4678a24977fbdcf04135) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:22.458533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:22.459986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62522c-9ef1-4346-8665-45799d80ccb3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:22.462911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:22.466548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 33/57] 10.222.0.47 () {56 vars in 1322 bytes} [Sat Jul 16 12:29:22 2022] GET /identity/v3/role_assignments?user.id=29ed4c3c304b4678a24977fbdcf04135&scope.domain.id=default&role.id=a495d38f67e447309647ac782785bd87 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:24.296292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce18f-7ed8-43d4-992d-97c0860eef8c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:24.296788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce18f-7ed8-43d4-992d-97c0860eef8c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:24.296869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce18f-7ed8-43d4-992d-97c0860eef8c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:24.298259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 25/58] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:24.303982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:24.304351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:24.304550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:24.321175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:24.322394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:24.387489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 34/59] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:24.411595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce18f-7ed8-43d4-992d-97c0860eef8c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:24.509201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce18f-7ed8-43d4-992d-97c0860eef8c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:24.531881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:24.542575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:24.543354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:24.543635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:24.543911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:24.545092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:24.546569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe697e2-89a4-41b0-8989-f0dbe3b52b38 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:24.548156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 26/60] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:24.554085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:24.649489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a302-3429-474e-9cc0-b39de03d2ede None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:24.677529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:24.682874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:24.683531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:24.683813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:24.684087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:24.684797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:24.686383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c15d89-7f19-4477-bbf6-61077da30409 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:24.774750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 35/61] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:24 2022] POST /identity/v3/users => generated 275 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:28.047374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-f0e5-4f41-a356-77c41fbce3bb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.047882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-f0e5-4f41-a356-77c41fbce3bb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.047991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-f0e5-4f41-a356-77c41fbce3bb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.050201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 27/62] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:28.055657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780009b-b29d-4fc7-b02b-81784fee6ab3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.056071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780009b-b29d-4fc7-b02b-81784fee6ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.056258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780009b-b29d-4fc7-b02b-81784fee6ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.062685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e263-01ef-451d-bc74-00992ed78617 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.063159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e263-01ef-451d-bc74-00992ed78617 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.063584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e263-01ef-451d-bc74-00992ed78617 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.065166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 28/63] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:28.071000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.071479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.071905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.092448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0780009b-b29d-4fc7-b02b-81784fee6ab3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:28.093926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0780009b-b29d-4fc7-b02b-81784fee6ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:28.106084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:28.107633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:28.171818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 36/64] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:28.178473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5d4e9-c753-4710-b4df-6d43c05320b5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.179016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5d4e9-c753-4710-b4df-6d43c05320b5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.179499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5d4e9-c753-4710-b4df-6d43c05320b5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.181162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 37/65] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:28.192710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 29/66] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:28.194406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.194776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.195152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.206171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.226038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:28.227473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:28.300445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 38/67] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:28.301308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.324623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b39a7-1b71-4e34-9438-2c0ff97a318a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.350560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.359890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.362846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.363824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.368080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.368080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.375069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9d7e3e11-bf5f-4c89-9bda-e5953cc0e40e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:28.383006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:28.383843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:28.385359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 30/68] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:28.386007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c1d-1373-4f15-9651-14e757927c4d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.386007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c1d-1373-4f15-9651-14e757927c4d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.386349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c1d-1373-4f15-9651-14e757927c4d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.389162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 31/69] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:28.389162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3c1d-1373-4f15-9651-14e757927c4d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.421875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1b3a4-8afd-43b0-915b-369628bfc511 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.459625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.460441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.460734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.461167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.473241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:28.482168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:28.483021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:28.483785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 39/70] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:28.484408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.505748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3c1d-1373-4f15-9651-14e757927c4d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.542483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.543281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.543568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.543853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.554597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7e383fd3-9d27-4b75-a2d9-168d48ed0f14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:28.562389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:28.563628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:28.565338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 32/71] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:28.570272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.570570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.570857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.588842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:28.593927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:28.594043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50f171-4b13-44df-b73b-623e262025f8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.601253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.628046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.629975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.630655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.630945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.631915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.632142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:28.632321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:28.633746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:28.643393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 40/72] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:28.644197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.659830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 33/73] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:28.661900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.745804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6f338-6787-47bb-8856-0f53ebaf39bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.758920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ded188-ce6b-42d0-aa63-bd0d4a522944 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.780109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.780850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.781168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.781449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.782188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:28.782497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:28.783926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553894f-56d7-4c9e-ab16-ab95221a1710 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:28.791611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 41/74] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:28.792831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2790-ff0c-42b4-8304-694f63a5fee2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.793146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2790-ff0c-42b4-8304-694f63a5fee2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.793429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2790-ff0c-42b4-8304-694f63a5fee2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.793556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.794374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.794814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.794913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 42/75] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:28.794983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.795355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea2790-ff0c-42b4-8304-694f63a5fee2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.795717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:28.796219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:28.797423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:28.806721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 34/76] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:28.808814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.825496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.844066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:28.897261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea2790-ff0c-42b4-8304-694f63a5fee2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.910371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af14668-5847-4995-a97c-4b415dfe1c97 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:28.930475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.931340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.931620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.931892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.933097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:28.934550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:28.936329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 43/77] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:28.937012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:28.947626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:28.948426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.948709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.948981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.959498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-af3065a7-d74f-49ad-8a39-c60d3fbf2d80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:28.967030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:28.967907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:28.968667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 35/78] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:28.969651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:28.969651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:28.969937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:28.987256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:28.988555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:29.033967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.037541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee16f6-87aa-40d6-b9db-d8d5004b3da2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.054792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 36/79] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:29.058568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.071954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.072835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.073226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.073504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.085621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.087147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.088604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.090472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 44/80] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:28 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.095137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.158659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4e88-c14f-4985-92a8-eb36732d173e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.191122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13397-e9ac-4378-80f0-f442f560d9c3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.193254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.194135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.194424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.194702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.195906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.197398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.199034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 37/81] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.203593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.224534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.225225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.225513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.225791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.250841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.289234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.292860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.295197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.297847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa93dcd-ebb9-4760-8d54-4215521873b2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.332580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.333201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.333486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.333763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.334504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:29.334803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.336179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.344236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 38/82] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.344886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.396927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 45/83] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:29.398867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.443241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d2eb-d008-4853-a68b-7f0ebb9af932 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.453368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.477886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.478680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.478963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.479239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.480416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.481875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.483469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 39/84] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.484217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.499118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1387d00-ef35-45d1-abb1-8a17620a7757 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.512481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.531276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.531918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.532201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.532478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.587624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8fcf0-16ed-45b3-9a72-48e09187579b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.600882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.602395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.621030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.621688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.621969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.622244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.655737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.688314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.690798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.699974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 46/85] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:29.702259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.714319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.790828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 40/86] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:29.791664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.799641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468c472-9c2f-4d9e-98df-96d79de1c50c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.833501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.834366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.834651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.834928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.836122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.837613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.843826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 47/87] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.843826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.866743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.888128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a617e1-22a0-4802-985e-c8e35b4c435e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.911340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:29.922455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.923308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.923803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.924255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.925647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.927084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:29.928876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 41/88] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:29.928876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:29.945675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31e792-e6a0-4cec-af36-e39994d5576a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:29.979967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:29.980632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:29.980918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:29.981239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:29.981946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:29.983379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.029292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae304265-a2d5-42f9-b887-9c5f0e3dd2f0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.061673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403664-0942-4608-b699-fc356de97aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.062539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403664-0942-4608-b699-fc356de97aec None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.062805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403664-0942-4608-b699-fc356de97aec None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.063080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa403664-0942-4608-b699-fc356de97aec None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.064775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa403664-0942-4608-b699-fc356de97aec None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.066322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa403664-0942-4608-b699-fc356de97aec None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.068006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 42/89] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:29 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.068754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403664-0942-4608-b699-fc356de97aec None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.083043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 48/90] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:29 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:30.084488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.139524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.182730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.193504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa403664-0942-4608-b699-fc356de97aec None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.203360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ee196-163e-431e-a948-be2db3ec8537 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.231774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.232641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.232928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.233227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.234976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.236489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.238266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 43/91] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.239973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.250742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.251579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.252384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.252620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.330050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.332122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.360694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72182ea9-22a3-4735-8e9f-db216d9d654d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.393690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.404338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.405101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.405386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.405662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.406508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=29ed4c3c304b4678a24977fbdcf04135) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:30.406820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.408354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.418645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 44/92] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/role_assignments?user.id=29ed4c3c304b4678a24977fbdcf04135&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.419526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.438709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.466237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 49/93] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:30.467366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.546973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ba9a6-a350-49ae-b194-8ce8882d25e6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.584438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca7a31-0de3-440a-8d96-2b88b2602921 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.586137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.586978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.587412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.587884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.588747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:30.591647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.591647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.602545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 45/94] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=99bae25deed04db6ab077873f28587b2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.603641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.705958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.706654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.707075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.707363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.708341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:30.708663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.710202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.716833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 50/95] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.727305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.814177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25610a-15f0-4e02-81d1-5411bfb61cd3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:30.858467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:30.859348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:30.859632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:30.859916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] PATH_INFO: `/v3/projects/2caece187ccc40b3887b66162b564214` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:30.861824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] RBAC: Authorizing `identity:get_project(project_id=2caece187ccc40b3887b66162b564214)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:30.863372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90993732-0a51-4f09-820e-f278d73c2bae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:30.865358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 46/96] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/projects/2caece187ccc40b3887b66162b564214 => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:30.871462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:30.925706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:30.985306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb919-d49e-4f37-a6ec-b6a2db31dfe2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:31.021479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:31.022155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:31.022449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:31.022736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:31.023589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=2caece187ccc40b3887b66162b564214, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:31.023917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:31.025537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cad91-0510-4430-8420-0a6d8dbd6130 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:31.036192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 51/97] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:30 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=2caece187ccc40b3887b66162b564214&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:32.612578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f802-1f1f-42b6-9606-8a0e8a9f6dd3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:32.615517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f802-1f1f-42b6-9606-8a0e8a9f6dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:32.615517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f802-1f1f-42b6-9606-8a0e8a9f6dd3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:32.615517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 47/98] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:32.620316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:32.620709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:32.620890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:32.658090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:32.659597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:32.743540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 52/99] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:32.783578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f802-1f1f-42b6-9606-8a0e8a9f6dd3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:32.918756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f802-1f1f-42b6-9606-8a0e8a9f6dd3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:32.952237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:32.967759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:32.968561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:32.968845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:32.969152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:32.989986 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:33.001973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:33.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:33.003667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 48/100] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:33.011631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:33.075467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:33.172106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5790ba-5f4b-4c9d-a1f2-790c3a9b400d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:33.222209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:33.222885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.223174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.223455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.224210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:33.224519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:33.225938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:33.234326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 53/101] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:33.242136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:33.397493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07ea15-90a0-4cdc-95eb-c3a049d51060 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:33.444286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:33.445114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.445401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.445675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] PATH_INFO: `/v3/users/53244f25df9a4af4a67349013af32931` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.469822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] RBAC: Authorizing `identity:get_user(user_id=53244f25df9a4af4a67349013af32931)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:33.471650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec8b5-734e-4de2-945f-8fa3f5081308 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:33.473436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 49/102] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:33 2022] GET /identity/v3/users/53244f25df9a4af4a67349013af32931 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:33.479070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:33.484335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c10c-5527-42d3-8850-cc72dd151278 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.484625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c10c-5527-42d3-8850-cc72dd151278 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.484907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c10c-5527-42d3-8850-cc72dd151278 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.486841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 50/103] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:33.492249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.492533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.492809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.510835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:33.512266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:33.549645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:33.582137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 51/104] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 16 12:29:33.618104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:33.714966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:33.717915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bb3e9-7f39-4307-918d-2f0dce9d2a75 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:33.770006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:33.770912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.771524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.771962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.792786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6194eeb-4999-44de-8488-09286f8cc99f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:33.837518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:33.838594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.839038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.839455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.841255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:33.843267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b53a-b210-4881-9cb3-6defe44c134b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:33.846695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 52/105] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:33.848167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2661e4-4972-4632-9096-6f9f7beee331 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.848592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2661e4-4972-4632-9096-6f9f7beee331 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.849002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2661e4-4972-4632-9096-6f9f7beee331 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.850413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 53/106] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:33.851589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41210cd1-ee3e-4da8-890c-c0afb8b0a244 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.852008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41210cd1-ee3e-4da8-890c-c0afb8b0a244 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.852415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41210cd1-ee3e-4da8-890c-c0afb8b0a244 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.853774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 54/107] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:33.857285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.857747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.858202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:33.865973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:33.867453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0444fa3-3995-4c26-9c5a-f94bb3a28d9f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:33.882340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:33.884368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:33.924622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:33.971208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:33.977083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 55/108] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:33.977083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:33.981825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 54/109] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:33.985130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f902c31-dff6-4174-acc5-4a33f73ccb8d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:33.985130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f902c31-dff6-4174-acc5-4a33f73ccb8d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:33.985130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f902c31-dff6-4174-acc5-4a33f73ccb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.003099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1f902c31-dff6-4174-acc5-4a33f73ccb8d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:34.004634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1f902c31-dff6-4174-acc5-4a33f73ccb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:34.083951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 55/110] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:34.084364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2378f0-a1ee-4034-aab3-a42f2043523c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.085346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354615-c961-4d82-b893-3102ac16cd1a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.085650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354615-c961-4d82-b893-3102ac16cd1a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.085936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb354615-c961-4d82-b893-3102ac16cd1a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.091286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 56/111] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:34.091286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb354615-c961-4d82-b893-3102ac16cd1a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.123146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.123814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.124102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.124379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.125113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:34.127092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:34.187267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb354615-c961-4d82-b893-3102ac16cd1a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.215548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 56/112] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:29:33 2022] POST /identity/v3/projects => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:34.215699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.216378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.227660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.228469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.228752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.229061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.230362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:34.231953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f37023-8d5c-4009-96ac-03799d5cc0f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:34.234011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 57/113] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:34.240870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c26f-edf7-4206-92ea-b47bb024204e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.241188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c26f-edf7-4206-92ea-b47bb024204e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.241474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c26f-edf7-4206-92ea-b47bb024204e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.243173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 58/114] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:34.244207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.244494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.244773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.249252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.262508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:34.263743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:34.339196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 59/115] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:34.340063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.362023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856340e-fb52-48d2-95bc-1d0988ab4a7e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.403971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.404794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.405104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.405461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.416834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:34.424903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:34.425826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:34.426670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 57/116] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:34.427317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.460895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98dcc6f-b824-4b0f-9c23-4f4aa9c72220 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.490192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.498576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.499488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.499771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.506351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.510906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:34.512327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ad88bffa-fe5f-41a2-9761-9d024758ec8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:34.520109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:34.521172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:34.522099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 60/117] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:34.522889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.522889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.523109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.538230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7dd41-8048-48a6-8323-325fe63f5d11 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.545130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:34.546321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:34.577441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.578208 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.578602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.578867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.579751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=53244f25df9a4af4a67349013af32931) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:34.580572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:34.582056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:34.587838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 58/118] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/role_assignments?user.id=53244f25df9a4af4a67349013af32931&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:34.597479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.631044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 61/119] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:34.632299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.742217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90ab72-ce6f-472f-9ed1-52edbd618484 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.780428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebf5d1-8d65-4927-b420-d66bf4e15eb4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.788225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.789143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.789523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.789875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.801614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:34.810385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:34.815831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 59/120] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:34.816609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.819392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.819801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:34.827457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.828143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.828435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:34.828717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:34.830310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:34.830436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:34.831888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:34.840646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 62/121] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:34.845127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:34.948413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e0fb4-e66f-49ca-b6b6-7366aae345e1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.973905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4aff5-1229-4356-a614-d7b08f6cafd9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:34.998542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:34.999539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:34.999931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.000570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.002621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:35.003500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.005357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.013792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.014909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.015343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.015766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.021575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 60/122] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.021575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.030784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:35.041333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:35.042250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 63/123] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:35.048035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.100958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.102750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.139179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05d52d-b742-49d8-9fd1-7c6b87aeb8fc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.167004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2dc3d-6c03-4905-a2a2-b585f0356501 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.182284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.182973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.183264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.183546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.184314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:35.184626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.186040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.196541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 61/124] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.197857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.213160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.214043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.214333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.214619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.215922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.217569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.219305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 64/125] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.220063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.307562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6561d8-c3fb-47da-82e5-95dea4e844ea None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.335519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.337060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f9f9-90de-47e4-933e-a628021d65c8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.343759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.345131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.345131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.345943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.347685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.348654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.349402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.351536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 62/126] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.356936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.380822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.382762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.382762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.382762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.383682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:35.384170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.385894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.397191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 65/127] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.398260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.488584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66c62-2f07-4e0c-a3d2-fc96f4a776f2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.517556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73433ee-ccd4-40db-b68b-350f490c6af0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.524949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.525888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.526178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.526458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.527669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.529160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.531136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 63/128] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.532052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.556277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.568061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 66/129] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.568507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.601330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.610387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.642828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813fcd5-635a-4076-b1c2-da5d6e4ae8d3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.674785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b27243-4a70-4223-bfe1-9ddd3b1b5ab8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.680158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.681188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.681580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.681960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.683748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.685016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.686877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 64/130] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.688824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.713131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.714313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.714805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.715258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.716647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.718331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.720184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 67/131] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.721933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.810358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20c4c5-4f41-482a-baec-5b92c6469a9b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.838654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1140e4-90cd-4602-a043-ca92a5bd0e1e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.842368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.847846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.848796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.849142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:35.849244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.849613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] PATH_INFO: `/v3/projects/2caece187ccc40b3887b66162b564214` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.850887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] RBAC: Authorizing `identity:get_project(project_id=2caece187ccc40b3887b66162b564214)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.852685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.854259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 65/132] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/projects/2caece187ccc40b3887b66162b564214 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:35.855161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.879625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:35.880853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:35.881379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:35.881842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:35.884398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, user_id=f3b3c2d0f7ab4159b27ebbffbf123f03, role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:35.886165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:35.961054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534e072-e227-4101-b821-e86f9b850c2c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:35.989851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 68/133] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:35 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/99bae25deed04db6ab077873f28587b2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:35.992949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:35.999210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.000191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.000580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.001135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.003708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, user_id=f3b3c2d0f7ab4159b27ebbffbf123f03, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.005289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.106696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 66/134] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:35 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:36.108768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb73c9-0a62-4cc4-9e37-6035cda0b602 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:36.112108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:36.149820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.151194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.151690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.152223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.156094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.158899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.161117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 69/135] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:35 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:36.170877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:36.211695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:36.220448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:36.260479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ab55-58c9-4a6e-9288-a05ee173a6ef None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:36.301728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.303017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.303565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.304048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] PATH_INFO: `/v3/projects/2caece187ccc40b3887b66162b564214/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.306500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] RBAC: Authorizing `identity:create_grant(project_id=2caece187ccc40b3887b66162b564214, user_id=f3b3c2d0f7ab4159b27ebbffbf123f03, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.308169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1652e-3650-4d86-a7dd-991fca26e160 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.333629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227d09f-90d8-4b77-b375-54c767f5ae98 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:36.377111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.378032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.378323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.378606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/29ed4c3c304b4678a24977fbdcf04135/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.380697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, user_id=29ed4c3c304b4678a24977fbdcf04135, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.382206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.404460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 67/136] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:36 2022] PUT /identity/v3/projects/2caece187ccc40b3887b66162b564214/users/f3b3c2d0f7ab4159b27ebbffbf123f03/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:36.405851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2141-f8ea-467b-bdb1-f3e0fe5044bb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.406140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2141-f8ea-467b-bdb1-f3e0fe5044bb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.406415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2141-f8ea-467b-bdb1-f3e0fe5044bb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.408801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 68/137] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:36.416852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.417168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.417450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.439058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:36.440364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:36.455578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 70/138] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:36 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/29ed4c3c304b4678a24977fbdcf04135/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:36.541980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 69/139] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:36.582930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:36.695649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:36.702464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79db1-78cd-42e2-ae18-c8cebece97ae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:36.746877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.747690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.747981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.748261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.753923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.755407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10a6f5-7478-4042-9d5f-aa9f604e5ba1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.757507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 71/140] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:36.763452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:36.824749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:36.870463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3915-fed8-4967-b4d1-03e18de2568d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.870990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3915-fed8-4967-b4d1-03e18de2568d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.871065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3915-fed8-4967-b4d1-03e18de2568d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.872407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 72/141] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:36.880220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.880501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.880785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.895394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c6aab-641e-4ba1-866f-a2df829ba544 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:36.899063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:36.900269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:36.960502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:36.961202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:36.961495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:36.961780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:36.962521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:36.964047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:36.983739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 73/142] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:37.011335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:37.090024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 70/143] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:36 2022] POST /identity/v3/users => generated 283 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:37.135529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:37.144751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90317a0-57c6-4f31-a850-e1e82565feb8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:37.209811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:37.210672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:37.210897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:37.211176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:37.230903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:37.239202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:37.240046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:37.240814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 74/144] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:37.247529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:37.389744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75b9f-ae4c-45e2-be4f-5bfc67c0be44 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:37.399686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:37.431993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:37.432742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:37.433025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:37.433347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:37.434103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:37.434409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:37.435807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:37.446843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 71/145] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:37.452321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:37.619643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97b5d3-16b1-4147-9272-e9d4fa814e7f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:37.659851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:37.660799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:37.661014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:37.661318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] PATH_INFO: `/v3/users/53244f25df9a4af4a67349013af32931` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:37.665764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=53244f25df9a4af4a67349013af32931)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:37.667242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:37.668913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 75/146] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:37 2022] GET /identity/v3/users/53244f25df9a4af4a67349013af32931 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:37.674837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:37.849073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bac45-27ac-4581-bb7b-dcc2e2ec4930 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:37.895891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:37.896689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:37.896971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:37.897284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:37.898463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:37.899920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff429b9-ecf3-4e2b-851b-2fdbed9f4093 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:37.901654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 72/147] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:37 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:37.907416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:37.951965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:38.060838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3b5b6-5384-4665-8253-9741f25de4a1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:38.104141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:38.105054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:38.105352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:38.105629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/53244f25df9a4af4a67349013af32931/roles/897f530f43b540ba9dc83bf620bc4e97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:38.108527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, user_id=53244f25df9a4af4a67349013af32931, role_id=897f530f43b540ba9dc83bf620bc4e97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:38.110289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6e966-eb68-447a-a319-1470b1e878c4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:38.192988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 76/148] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:37 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/users/53244f25df9a4af4a67349013af32931/roles/897f530f43b540ba9dc83bf620bc4e97 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:38.889586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc8a0a-dc1a-447b-a611-20f6d3b3cc4a None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:38.890194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc8a0a-dc1a-447b-a611-20f6d3b3cc4a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:38.890194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc8a0a-dc1a-447b-a611-20f6d3b3cc4a None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:38.891823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 73/149] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:38.905867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964accfa-0409-439f-8369-edb589a1749e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:38.906502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964accfa-0409-439f-8369-edb589a1749e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:38.907003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964accfa-0409-439f-8369-edb589a1749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:38.930084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-964accfa-0409-439f-8369-edb589a1749e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:38.932049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-964accfa-0409-439f-8369-edb589a1749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:39.016610 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 77/150] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:39.060046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc8a0a-dc1a-447b-a611-20f6d3b3cc4a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.115010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.128692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef8a1-1cb5-49fd-8d95-e2d99eb217cb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.133087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef8a1-1cb5-49fd-8d95-e2d99eb217cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.133087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef8a1-1cb5-49fd-8d95-e2d99eb217cb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.133087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 78/151] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:39.145675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.146465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.147277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.190209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:39.191785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:39.195727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.249080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc8a0a-dc1a-447b-a611-20f6d3b3cc4a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.289565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 79/152] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:39.304544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.305399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.305681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.305954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.323848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.326805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:39.337063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:39.337997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:39.338801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:39.341569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 74/153] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:39.342581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.454413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849ed7c-5a89-492a-919e-417eb8a90f38 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.476601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9e1c-e581-44e4-84f5-cbe85ebf3a01 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.494482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.495535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.498158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.498158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.505227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.508826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:39.516806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:39.518162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:39.519008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:39.521596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 80/154] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:39.527665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.541818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.542519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.542807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.543161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.543887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:39.544238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:39.545675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:39.563293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 75/155] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:39.569497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.675368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5583-6679-4ccb-9930-2a5e1acbc0bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.725411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.726460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.727011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.727491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.728452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:39.728990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:39.730729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:39.744852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 81/156] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:39.753316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.756904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed8825-7dd4-4e2e-850e-45c0665183a5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.828098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.828994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.829342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.829624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.832499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:39.837060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:39.837060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 76/157] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:39.841260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:39.847763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.887022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:39.915250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888e79-8f78-4147-a17b-df91b98acf7d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:39.983920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:39.984790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.985155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.985529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:39.987579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:39.992821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca2499-b7b9-4200-8aaa-38e035d0343c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:39.994930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 82/158] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:39.996230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c65f5-d496-4da9-b3e6-b45200e7707d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:39.996687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c65f5-d496-4da9-b3e6-b45200e7707d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:39.997061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c65f5-d496-4da9-b3e6-b45200e7707d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.002627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 83/159] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:40.003456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c65f5-d496-4da9-b3e6-b45200e7707d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:40.004052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf7bb2-1699-4c50-9edb-1a427411f36d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:40.050293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:40.051534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.052019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.052558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.190433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:40.191924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e27fa-9366-4890-aa7b-be502e85694d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:40.193720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c65f5-d496-4da9-b3e6-b45200e7707d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:40.249764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:40.250395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:40.266088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:40.270255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.270255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.270255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.367981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 77/160] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:29:40.369332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f09db-bb85-4bd9-9922-aacbd2df1d89 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.369628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f09db-bb85-4bd9-9922-aacbd2df1d89 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.369909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f09db-bb85-4bd9-9922-aacbd2df1d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.406790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-142f09db-bb85-4bd9-9922-aacbd2df1d89 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:40.408037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-142f09db-bb85-4bd9-9922-aacbd2df1d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:40.409358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:40.410854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4b06-82bf-446f-bb94-9f3d3b24fd31 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:40.519483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 78/161] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:40.520807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28db12-a2ef-4301-b499-deecf28465dd None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.521126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28db12-a2ef-4301-b499-deecf28465dd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.521409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28db12-a2ef-4301-b499-deecf28465dd None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.522945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 79/162] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:40.523721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28db12-a2ef-4301-b499-deecf28465dd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:40.565742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 84/163] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:40.567046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.567333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.567627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.598016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:40.599344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:40.631496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:40.647671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:40.706409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 85/164] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:40.707273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:40.778090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28db12-a2ef-4301-b499-deecf28465dd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:40.832503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:40.833500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:40.833792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:40.834071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:40.835401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:40.836871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:40.838633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 80/165] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:40 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:40.839433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:40.872897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a01a6-a5e2-4014-b158-6085580cc60b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:40.978624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed09259-5766-4d71-b1b2-d6bcdeafdecc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.001805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.002621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.002906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.003176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.003752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:41.029392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.030188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.030468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.030740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.037751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:41.041684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:41.043140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:41.044873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 81/166] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:40 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:41.045675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:41.050899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:41.051774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:41.052794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 86/167] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:41.053491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.101394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:41.190317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e67c-9c96-4de9-84f1-10ad18ffd01b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.256241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9609a2-848b-44b0-88bd-b24e64e99c6e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.260639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.261478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.261753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.262027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.284608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:41.292176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:41.293070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:41.293865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:41.305591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 82/168] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:41.306420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.324895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.325766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.326046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.326314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.327141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:41.327447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:41.328873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:41.342246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 87/169] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=99bae25deed04db6ab077873f28587b2 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:41.343053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.425427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:41.529456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12968ae1-d43d-497c-80bb-1127fa0f2310 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.588766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:41.604136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.604816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.605131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.605493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.606336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:41.606648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:41.608159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:41.622673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 83/170] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:41.623533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.644913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c638e0-cff2-4ab9-a430-807d075a93d2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.769815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.770499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.770785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.771060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.771831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:41.772137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:41.773560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9719-2207-4cc1-a4c5-16b1d2a8f226 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:41.788725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 88/171] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:41.790113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1a3b-8430-4605-81ee-5fd747e32c31 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.790403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1a3b-8430-4605-81ee-5fd747e32c31 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.790679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1a3b-8430-4605-81ee-5fd747e32c31 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.792064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 89/172] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:41.797429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1a3b-8430-4605-81ee-5fd747e32c31 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:41.841325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2859-b094-4ed8-9dfd-4b4008d2431c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:41.881414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:41.924031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:41.924832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.925263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.925551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.926320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:41.926640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:41.928048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0177d87-3577-4cda-9365-2694eb1668c8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:41.944396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 84/173] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:41.945724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:41.945979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:41.946256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:41.977306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:41.978653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:42.007711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1a3b-8430-4605-81ee-5fd747e32c31 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.049336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.050471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.050890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.051300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] PATH_INFO: `/v3/users/29ed4c3c304b4678a24977fbdcf04135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.052807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] RBAC: Authorizing `identity:get_user(user_id=29ed4c3c304b4678a24977fbdcf04135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.055275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.057483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 90/174] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:41 2022] GET /identity/v3/users/29ed4c3c304b4678a24977fbdcf04135 => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:42.058858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.097512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.138443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 85/175] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:42.139470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.172313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6e051-85a3-4c86-86ab-f695f5809f20 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.215444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.216667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.217204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.217671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] PATH_INFO: `/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.219143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] RBAC: Authorizing `identity:get_user(user_id=f3b3c2d0f7ab4159b27ebbffbf123f03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.221707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.223731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 91/176] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/users/f3b3c2d0f7ab4159b27ebbffbf123f03 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:42.224723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.326266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15fe73-8e30-42e8-bcac-b01ad43f0ab8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.332394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c09be-142e-4c0c-8e64-72de835f4dfc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.338587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.340463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.364226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.364881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.365189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.365459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.367000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.367830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.368110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.368381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.378876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:42.386554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:42.387451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:42.388277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 92/177] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:42.388959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.433368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.434827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad87be-a0d9-4b71-ac90-fc2712337e7e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.489696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e909d4-957e-4240-ad7b-166170522e0b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.522451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.523091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.523374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.523656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 86/178] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d05-a57f-4112-be4b-73be410cc8da None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d05-a57f-4112-be4b-73be410cc8da None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d05-a57f-4112-be4b-73be410cc8da None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 87/179] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02ec1-cfde-4ea8-baeb-980cc76677e0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.537195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02ec1-cfde-4ea8-baeb-980cc76677e0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.538252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02ec1-cfde-4ea8-baeb-980cc76677e0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.538326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.538469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 88/180] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:42.539711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096b06-a406-4256-baf7-0837b578d393 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.539918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096b06-a406-4256-baf7-0837b578d393 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.540191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62096b06-a406-4256-baf7-0837b578d393 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.541580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 89/181] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:42.545979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62096b06-a406-4256-baf7-0837b578d393 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.557412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.598249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.599713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.651150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62096b06-a406-4256-baf7-0837b578d393 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.685216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.685882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.686166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.686440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.687183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:42.687488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.688943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859525a9-103f-40f6-a77f-9c6c193a5307 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.696499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 93/182] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:42.697357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.697492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 90/183] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:42.698620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc8dc-6b92-47e3-b94f-ac2032f57b8b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.698916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc8dc-6b92-47e3-b94f-ac2032f57b8b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.699196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc8dc-6b92-47e3-b94f-ac2032f57b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.717464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5bdbc8dc-6b92-47e3-b94f-ac2032f57b8b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:42.718680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5bdbc8dc-6b92-47e3-b94f-ac2032f57b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:42.742024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.780523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.785584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 91/184] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:42.786669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.786957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.787245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.804421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:42.805634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:42.810718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089165f0-e780-4fe7-8616-837fc46176cc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:42.843822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:42.845261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.845505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.845772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.847208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:42.848652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91161dc3-d19c-47c9-b095-cd4314fe7a09 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:42.850365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 94/185] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:42.851479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:42.851758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:42.852027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:42.867669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:42.868824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:42.884351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 92/186] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:42.884351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.944164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 95/187] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:42.944926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:42.986189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:42.992318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1bd78-48dc-4492-b7ab-990edce7adde None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.020525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.026298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.027157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.027439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.027710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] PATH_INFO: `/v3/projects/2caece187ccc40b3887b66162b564214` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.028891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] RBAC: Authorizing `identity:get_project(project_id=2caece187ccc40b3887b66162b564214)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.030407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.037770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 93/188] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/projects/2caece187ccc40b3887b66162b564214 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.037770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.043391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c607e-7f10-4a36-a6ad-f1fe7870a95b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.076832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.077722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.077999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.078266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] PATH_INFO: `/v3/users/53244f25df9a4af4a67349013af32931` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.080225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] RBAC: Authorizing `identity:get_user(user_id=53244f25df9a4af4a67349013af32931)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.081689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.083342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 96/189] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:42 2022] GET /identity/v3/users/53244f25df9a4af4a67349013af32931 => generated 289 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.084086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.133362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd352-4c92-415b-9df1-9b993c803674 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.165902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.166780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.167060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.167331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.180685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:43.182530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edad5b1-d28c-4c56-8748-6422a49e0812 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:43.189376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:43.190252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:43.191043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 94/190] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:43.191694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.192669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.220006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.220637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.221202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.221480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.222313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=29ed4c3c304b4678a24977fbdcf04135) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:43.222625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.224064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.230145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 97/191] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/role_assignments?user.id=29ed4c3c304b4678a24977fbdcf04135&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.231044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.252125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.297089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087de250-82e5-4837-bd87-705d2081d79a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.335779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.336597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.336877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.337605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.348905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:43.351486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ef68-0ef6-4892-b25d-5640a4600c88 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:43.356619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:43.357570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:43.358386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 95/192] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:43.359626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.396082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.396945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.397265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.397542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.398730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.400169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.401992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 98/193] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.402589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.453420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.479673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c83c82-386a-432f-b09e-75df99ae5e03 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.496983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.513090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5a18-764f-4435-a1f4-88b77d40dace None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.513865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=2caece187ccc40b3887b66162b564214, user.id=f3b3c2d0f7ab4159b27ebbffbf123f03) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.519259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.523924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 96/194] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/role_assignments?user.id=f3b3c2d0f7ab4159b27ebbffbf123f03&scope.project.id=2caece187ccc40b3887b66162b564214&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.530489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.550120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.550789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.551071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.551347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.632036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.633590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.663462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2122-e665-42ac-9132-a0702d46c6d6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.679954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.706241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.707173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.707173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.707467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.708222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:43.708528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.710007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.721808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 97/195] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.721808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.757266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 99/196] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:43.758956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.772640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.826801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5201329-cdab-4a94-b09c-0e59477442b5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.854782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392286fd-ad40-4fcd-9af9-ff2a3ce593ae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.859761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.860446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.860730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.861004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.861774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:43.862083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.863450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.871392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 98/197] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:43.872170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:43.895743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:43.896412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:43.896696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:43.896977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:43.897719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:43.899157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:43.908806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:43.968287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837341-c312-4691-a04c-566fda91f091 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:43.975367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.001721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.002599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.002872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.003144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.003398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 100/198] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:43 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:44.004202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.018108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.019531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.026156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 99/199] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:43 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:44.026156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.119252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef31c7-6e87-4503-843b-6701e53570f4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.146869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226356cc-0ee8-4ee4-8fe6-3d8f61ecbdcc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.159459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.160332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.160616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.160887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.162125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.163567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.165324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 101/200] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:44.170197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.196959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.197875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.198159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.198531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.199736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.201234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.202896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 100/201] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:44.203655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.241759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.245313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.297362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fef8e-2267-433d-927c-8f06f32aaa5b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.336137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818d6e-cd5f-4b72-976f-aeed9ddbc1ca None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.339783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.340454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.340737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.341013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.376959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.377670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.377973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.378229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.379074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, scope.project.id=fa56928f7a294120ac8fe81e41c3a013, user.id=53244f25df9a4af4a67349013af32931) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:44.379390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.380905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.387095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 101/202] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/role_assignments?user.id=53244f25df9a4af4a67349013af32931&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:44.387916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.442131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.443901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.572134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.576088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.577447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d7763-49f2-4463-a105-5aa1d0b74f42 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.613001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.613691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.613976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.614251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.624274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 102/203] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:44.635443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.694026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.695506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.762126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9ed72-56d4-4fcb-80e4-64ad74b8dc12 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.804065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.804932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.805252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.805526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.806790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:44.808845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:44.815676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 103/204] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:44.816982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.834048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.842544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 102/205] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:44.848535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:44.865118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:44.955411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7aa-7ea0-47de-a988-841f063bbe60 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.959392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab578744-2932-4e7e-8268-8845d22d1c1b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:44.996388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.997327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:44.997713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:44.998161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:44.999111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=8f36550d373b45d195e4431dff9b8a97) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:44.999307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:44.999610 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:45.000463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:45.000992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:45.001631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:45.001743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab95a4-def1-4b74-949f-84a6e0da31a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:45.003073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:45.005535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:45.007703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 103/206] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:45.012515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 104/207] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:44 2022] GET /identity/v3/role_assignments?user.id=8f36550d373b45d195e4431dff9b8a97&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=99bae25deed04db6ab077873f28587b2 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:45.013716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:45.087940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:45.129842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b03674-7e38-4231-bfb9-e96ac448bd0c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:45.171306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:45.172307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:45.172798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:45.173382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:45.174411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=8f36550d373b45d195e4431dff9b8a97) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:45.174727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:45.177337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fd78f-0281-4628-9cef-6b42ac179399 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:45.184228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 104/208] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:45 2022] GET /identity/v3/role_assignments?user.id=8f36550d373b45d195e4431dff9b8a97&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:46.242058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326de60-7b0a-4ba6-8ac2-fc7b21bf62bf None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.242552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326de60-7b0a-4ba6-8ac2-fc7b21bf62bf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.242659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326de60-7b0a-4ba6-8ac2-fc7b21bf62bf None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.244129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 105/209] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:46.249214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.249606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.249770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.267582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:46.268772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:46.340067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198ca8-0f0d-4c44-a061-842d968d76bd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.340386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198ca8-0f0d-4c44-a061-842d968d76bd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.340666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198ca8-0f0d-4c44-a061-842d968d76bd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.342350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 106/210] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:46.347630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.347920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.348205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.349289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 105/211] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:46.369428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:46.370963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:46.372972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:46.431525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:46.457080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 107/212] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:46.477180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:46.483332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535bfe-a2dd-4667-b895-2dc346b73842 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:46.509323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:46.523691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:46.525335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.525823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.526321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.540620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:46.551791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:46.552749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:46.560806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 106/213] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:46.562228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:46.598970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259316af-4293-47ca-8e17-1336b0bf3aa2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:46.637585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:46.638381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.638714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.639036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.640320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:46.642084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:46.648722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 108/214] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:46.649668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:46.685767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67f57-6bd7-4155-929a-8a464f7d486c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:46.705228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:46.727114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:46.727797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.728084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.728361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.729401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:46.729711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:46.731881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:46.749124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 107/215] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:46.751161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:46.774206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:46.782113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8f603-b96b-46a6-ae92-b52ba53cf0ff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:46.825715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:46.826391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.826673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.827210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.827947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:46.830760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:46.880452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c71b-36bf-4ecb-967d-6ed3b8ca8ac4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:46.916391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:46.917685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:46.918176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:46.918666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] PATH_INFO: `/v3/users/49602335126a4413adefdb964e9d6728` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:46.934725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] RBAC: Authorizing `identity:get_user(user_id=49602335126a4413adefdb964e9d6728)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:46.936275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883063b-0d03-4ef0-9807-eb7cf16e49d2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:46.937723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 109/216] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:46 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:46.948725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 108/217] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:46 2022] GET /identity/v3/users/49602335126a4413adefdb964e9d6728 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:46.950212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.055662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d167d-632b-4c46-a7a7-93df09d923be None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.055992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d167d-632b-4c46-a7a7-93df09d923be None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.056226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d167d-632b-4c46-a7a7-93df09d923be None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.057843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 109/218] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:47.064021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.064636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.065151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.073914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293a5a9-3340-4ba0-b2bc-b5ccdc628062 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.084095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:47.085691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:47.112799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.113501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.113809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.114076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.122352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.135984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.164280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 110/219] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:47.190894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.206758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:47.208322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5d3a3-0a36-482a-b4c4-57f039772149 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:47.330048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb021c2-7150-452c-832b-daa58072e42a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.336358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 110/220] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:47.337738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd03cc6-5088-4768-97e6-f98950392c84 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.338029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd03cc6-5088-4768-97e6-f98950392c84 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.338309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd03cc6-5088-4768-97e6-f98950392c84 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.339592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 111/221] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:47.342162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd03cc6-5088-4768-97e6-f98950392c84 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.372645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.373492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.373772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.374040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.386894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e63ab40d-dad3-443d-a69a-241f8c434a6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:47.397005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:47.397971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:47.398765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 111/222] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:47.400021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.400301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.400574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.413020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.422526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:47.424615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:47.455916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.483669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd03cc6-5088-4768-97e6-f98950392c84 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.521063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 112/223] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:47.521063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.545758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.547123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.547748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.548273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.549880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:47.551554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:47.553649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 112/224] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:47.554640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.642257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecab8fc-b89b-4392-a0fe-74129b739b2d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.663367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ede8-ee16-41d6-9a09-205ee475c09b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.687729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.688722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.689009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.689340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.690420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:47.690728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:47.692360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:47.701380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.702506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.702990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.703274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 113/225] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:47.703667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.704733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.705419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.718923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:47.727551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:47.728426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:47.729276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 113/226] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:47.729965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.736708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:47.847168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217bb57-f32e-4398-aa99-360f061b4545 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.852219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc71cd0-b48d-447b-b24a-c812974f93ff None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:47.892902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.893802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.894083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.894358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.896743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:47.898242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:47.900053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 114/227] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:47.903534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:47.904512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:47.904997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:47.905288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:47.905807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:47.906904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=49602335126a4413adefdb964e9d6728) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:47.907424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:47.910472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:47.918048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 114/228] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/role_assignments?user.id=49602335126a4413adefdb964e9d6728&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:47.919206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:48.025458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:48.027051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:48.047080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2f10-4c5c-45f7-bec0-4c8915987b45 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.089775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.091892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.092439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.092962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.095013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.097477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.100653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 115/229] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:48.111179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:48.164953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a16b7-0a3c-4d2b-9aba-9db9ee819fde None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.205015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.205729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.206012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.206291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.207055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:48.207364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.208866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.219262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 115/230] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:48.225418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:48.249016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d6f8-2869-4157-83a3-a8dec2c737aa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.291405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.292662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.293187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.293893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/8f36550d373b45d195e4431dff9b8a97/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.296625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, user_id=8f36550d373b45d195e4431dff9b8a97, role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.299010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.380102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 116/231] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:48 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/8f36550d373b45d195e4431dff9b8a97/roles/99bae25deed04db6ab077873f28587b2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:29:48.389853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299504a-ead4-485d-aaa1-00dd62aaf2ac None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.402995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:48.431980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.433281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.433769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.434248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.435848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.437691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.447126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 116/232] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:48 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:48.452919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:48.474307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:48.584183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55613ac-0fca-4a87-afaf-8e1012ec45f4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.622691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.623996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.624483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.624961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.627223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.628870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727298-dd4c-4289-a27e-e626b251c80a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.630960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 117/233] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:48 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:48.637080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:48.752156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b61e3-21a3-4aa3-ab93-fb0646e27530 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:48.789810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:48.790973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:48.791401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:48.791819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/8f36550d373b45d195e4431dff9b8a97/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:48.793993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, user_id=8f36550d373b45d195e4431dff9b8a97, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:48.795598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d542a74-d51f-4035-bace-6b6e1ca1dfbc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:48.916205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 117/234] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:48 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/8f36550d373b45d195e4431dff9b8a97/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:49.395113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00146-ba5e-4533-9235-b93f2f050f03 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.395626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00146-ba5e-4533-9235-b93f2f050f03 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.395732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00146-ba5e-4533-9235-b93f2f050f03 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.397378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 118/235] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:49.402280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.403068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.403529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.426055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:49.427548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:49.445448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0158-62f9-48ab-8a1a-530a0995492b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.446055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0158-62f9-48ab-8a1a-530a0995492b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.446527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f0158-62f9-48ab-8a1a-530a0995492b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.448120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 119/236] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:49.453926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.454487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.454956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.470386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:49.475536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:49.477066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:49.541031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 118/237] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:49.565135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 120/238] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:49.585942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:49.597454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:49.621432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:49.738162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651193ee-f87c-492d-a138-dba4b569c3dc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:49.783096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:49.784408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.784891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.785408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.787584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:49.790731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:49.794985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 119/239] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:49.805274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:49.813867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e36292-f0b5-4836-af35-c3cecaa72bb7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:49.867564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:49.911116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:49.911927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:49.912211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:49.912488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:49.935566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:49.950681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f58c-3164-4b89-8869-b9df28053e22 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:49.966778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:49.967733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:49.968517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 121/240] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:49.975341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.009876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:50.010489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.010772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.011039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.011771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:50.013265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:50.076331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:50.105826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 120/241] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:49 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:29:50.124103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7f29-b031-4140-a576-bac79969d4a0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:50.194468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:50.195161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.195445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.195725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.196649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:50.196942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:50.198406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:50.208544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 122/242] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:50.214133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.347246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:50.357427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56a8b-49f3-4de7-98e7-73be53d6dfe4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:50.398931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:50.399821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.400365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.400641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] PATH_INFO: `/v3/users/a720729a2a2f4ef8910efeeec32d237b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.416188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] RBAC: Authorizing `identity:get_user(user_id=a720729a2a2f4ef8910efeeec32d237b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:50.417794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d355b-7e2c-4a67-983b-759aa93f8d3a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:50.419847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 121/243] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:50 2022] GET /identity/v3/users/a720729a2a2f4ef8910efeeec32d237b => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:50.425554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.470909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4df025-45e8-4df2-bfc9-9cb07a841aaa None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.471241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4df025-45e8-4df2-bfc9-9cb07a841aaa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.471599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4df025-45e8-4df2-bfc9-9cb07a841aaa None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.473398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 122/244] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:50.482394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.482669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.483240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.512474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:50.515889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:50.566399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ca642-bae0-477c-b447-0ee77dfb9bad None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:50.615814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:50.621708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:50.622373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.622655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.622931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.623697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 123/245] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:50.675005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.704420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:50.706062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:50.773554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:50.803454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fe12b-bde6-4df4-a4e6-2bfbe078c8e8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:50.825344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 123/246] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:50.831011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.847246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:50.848048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:50.848329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:50.848603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:50.862463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:50.863970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:50.876207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:50.877253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:50.878043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 124/247] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:50.883292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:50.961816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b45d2-8837-4bf3-86fc-15728a88f007 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.028712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.029611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.029892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.030167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.033000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.034419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.041945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 124/248] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:50 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:51.045341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.062676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5442a42-5028-48f8-a176-4bdde39a52b2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.132067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.132669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.132952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.133267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.134022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:51.134329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.135792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.148616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:51.164005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 125/249] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:51.181348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.258390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:51.261801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e838-bf4d-4707-a6ba-29f06efcbdc1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.320716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d0d-7928-4347-b14d-e5ac08332557 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.353747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.354419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.354700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.354972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.355843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=a720729a2a2f4ef8910efeeec32d237b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:51.356139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.357643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17438d7d-573b-462a-b884-6be69d22c1cb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.361150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.362195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.362620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.363042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] PATH_INFO: `/v3/users/49602335126a4413adefdb964e9d6728` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.364420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] RBAC: Authorizing `identity:get_user(user_id=49602335126a4413adefdb964e9d6728)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.366317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.367835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 125/250] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:51 2022] GET /identity/v3/role_assignments?user.id=a720729a2a2f4ef8910efeeec32d237b&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:51.368569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 126/251] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:51 2022] GET /identity/v3/users/49602335126a4413adefdb964e9d6728 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:51.373134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54fcc-523a-43b9-8713-e4995b5083ed None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.373134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54fcc-523a-43b9-8713-e4995b5083ed None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.373134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54fcc-523a-43b9-8713-e4995b5083ed None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.374870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 126/252] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:51.375217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.390806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.391082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.391904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.433187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:51.434394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:51.469595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:51.569244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a11c-f8d4-4e13-956f-1455c4a92b2e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.614799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 127/253] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:51.644834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.645652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.645935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.646208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.650103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.653852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.655340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.657765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 127/254] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:51 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:51.665496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.827902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:51.839539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef46d5-a378-4f10-8757-a155e4d34089 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.884785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9859c-8b2f-47b0-bd51-8bf66216fb4c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:51.902581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.903318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.903596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.903864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.926990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:51.935967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:51.936900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:51.937721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 128/255] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:51.945850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:51.981766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:51.982671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:51.982958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:51.983238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/49602335126a4413adefdb964e9d6728/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:51.985827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, user_id=49602335126a4413adefdb964e9d6728, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:51.987272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c0037-18e2-4cf6-a112-a26278052af1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:51.994036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:52.073392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050c97d-1460-4873-acd0-2ecc372e5c64 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:52.107717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 128/256] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:51 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/49602335126a4413adefdb964e9d6728/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:52.109591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747a6-fa74-4747-9b1c-406cc66dc987 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.110763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747a6-fa74-4747-9b1c-406cc66dc987 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.111302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747a6-fa74-4747-9b1c-406cc66dc987 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.111574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:52.112317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.112595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.112919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.113720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:52.114028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:52.114333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 129/257] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:52.120121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:52.123592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.124114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.124590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.130692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 129/258] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:52.139762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:52.169177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:52.170745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:52.191089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:52.339236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03e54-d46f-49c2-920f-00e6127ce1c0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:52.367426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 130/259] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:52.403624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:52.404557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:52.405489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.405804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.406084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.408491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:52.409615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:52.411679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 130/260] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:52.417410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:52.508013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:52.528407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856dda8-4b3e-425b-a477-16bb82c393d1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:52.579286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:52.579955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.580239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.580509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.629673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a23cf-2450-4ac1-9358-cd36dcab6538 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:52.702646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:52.710408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:52.712151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:52.724484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:52.725639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.726125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.726596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.757916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:52.771730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:52.772619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:52.773483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 131/261] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:52.777517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:52.860412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:52.885757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 131/262] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:29:52.894073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:52.917502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba505a8-8afb-4ff6-b11f-ef79694ebe64 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:52.954197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:52.954869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:52.955148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:52.955418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:52.956163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:52.956461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:52.957965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:52.966484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 132/263] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:52.973767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:53.113396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:53.123227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8d92-6296-47b4-933e-3dc95262acc3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:53.203445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:53.204463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.204738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.205005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.206370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:53.208162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:53.214941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 132/264] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:53.215771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:53.252443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76466b-e520-46d9-b910-7dc93b8c40cd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:53.309321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:53.336512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:53.337799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.338287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.338759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] PATH_INFO: `/v3/users/8f36550d373b45d195e4431dff9b8a97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.340177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] RBAC: Authorizing `identity:get_user(user_id=8f36550d373b45d195e4431dff9b8a97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:53.341864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:53.349292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 133/265] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:52 2022] GET /identity/v3/users/8f36550d373b45d195e4431dff9b8a97 => generated 283 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:53.354152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:53.404326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb62e76-470d-47d1-b108-2151bdfa47c4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:53.468373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:53.469085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.469364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.469641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.470740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=8f36550d373b45d195e4431dff9b8a97) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:53.471054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:53.472494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb847-467a-4ae1-9318-01fc642a33b6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:53.485720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 133/266] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:53 2022] GET /identity/v3/role_assignments?user.id=8f36550d373b45d195e4431dff9b8a97&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=99bae25deed04db6ab077873f28587b2 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:53.487036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676071ea-eaea-423e-b024-8477b435ae6f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.487329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676071ea-eaea-423e-b024-8477b435ae6f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.487610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676071ea-eaea-423e-b024-8477b435ae6f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.488958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 134/267] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:53.494744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366c76-a6c7-46e5-a304-43feb0d1f6b3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.495052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366c76-a6c7-46e5-a304-43feb0d1f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.495300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366c76-a6c7-46e5-a304-43feb0d1f6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.526173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cd366c76-a6c7-46e5-a304-43feb0d1f6b3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:53.528196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cd366c76-a6c7-46e5-a304-43feb0d1f6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:53.534211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fc44-cba5-4f0f-bc0a-16202f355998 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:53.585151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:53.634945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:53.635716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.635887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.636154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.679885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 135/268] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:53.682368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab8c3-dd55-4f2d-82e7-8af4352b87f3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.682856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab8c3-dd55-4f2d-82e7-8af4352b87f3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.683308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab8c3-dd55-4f2d-82e7-8af4352b87f3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.684771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 136/269] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:53.693946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.694240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.694518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.713663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:53.714863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:53.756083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:53.758718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:53.763209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:53.790812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 137/270] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:53.792037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:53.907118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 134/271] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:53.911798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:53.923866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a4bb8-0864-41dc-88b2-9bd87e5e4808 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:53.929650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:53.967518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:53.968256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.968541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.968818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.980431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-75038358-5db4-4663-9774-bb293f4e9acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:53.988807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:53.989787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 138/272] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:53.995509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e6d93-75d9-41af-af3f-191c79ec84c3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:53.996220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e6d93-75d9-41af-af3f-191c79ec84c3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:53.996220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e6d93-75d9-41af-af3f-191c79ec84c3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:53.996220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 139/273] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:53.996220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e6d93-75d9-41af-af3f-191c79ec84c3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.033320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3f2ee-9837-4ae7-85e3-7a6f5835f89f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.038761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.070754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.071909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.072450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.072998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.074379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.076017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.077845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 135/274] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.078916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.112039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e6d93-75d9-41af-af3f-191c79ec84c3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.147622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.148670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.149124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.149559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.150896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.152470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f88ea-36fd-4788-bb85-69746d42762f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.159597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 140/275] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:53 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.161006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.161696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.162113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.165949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.186112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b73a-195d-4297-99aa-d8e7049f98bd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.186772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:54.189777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:54.224376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.224973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.225287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.225564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.226309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:54.226692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.228013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.236849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 136/276] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.237445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.273072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 141/277] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:54.273072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.309975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.349857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc62c-d3aa-427f-a5bf-36a872ed78ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.380803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6350-8561-4243-83bb-8904de0b9360 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.385184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.385865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.386234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.386523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.387239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.389157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.415996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.416658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.416944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.417254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.418112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=8f36550d373b45d195e4431dff9b8a97) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:54.418428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.419855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.426356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 142/278] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/role_assignments?user.id=8f36550d373b45d195e4431dff9b8a97&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.428202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.480237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 137/279] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:29:54 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:54.481804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.487418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.591490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.599953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe34ae-55d8-4ae5-96bb-561e930a6d5f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.671802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.673244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.673529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.673798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] PATH_INFO: `/v3/users/739d41fecbf44132aade8da48d968b22` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.685476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0d6d8-6d62-4bc0-a4d0-7cb1c3434aea None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.695760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] RBAC: Authorizing `identity:get_user(user_id=739d41fecbf44132aade8da48d968b22)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.696034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.698166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 143/280] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/users/739d41fecbf44132aade8da48d968b22 => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.704587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.729647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.731023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.731607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.732084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.745861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:54.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:54.756244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:54.757096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 138/281] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:54.763171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.847968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:54.852396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba36fc-f6d1-4c01-a070-43f04fffd44b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.895103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cd889-0da4-4589-82a4-bf31fff7a335 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:54.898829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.899215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.899501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.899772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.900742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, user.id=739d41fecbf44132aade8da48d968b22, scope.system=all) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:54.901093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.902569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0058a1-8f77-40cc-9155-12b1d07cf8b4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.910081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 144/282] 10.222.0.47 () {56 vars in 1308 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/role_assignments?user.id=739d41fecbf44132aade8da48d968b22&scope.system=all&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.923727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ea3-7f88-4d03-bacc-0d739f9ecdc1 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.924032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ea3-7f88-4d03-bacc-0d739f9ecdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.924299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496ea3-7f88-4d03-bacc-0d739f9ecdc1 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.926057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 145/283] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:54.931541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.931866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.932066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.941753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:54.942466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:54.942891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:54.943178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:54.943964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:54.944277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:54.945763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:54.954851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:54.956080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:54.956728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 139/284] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:54.962033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:54.982465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.045151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 146/285] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:55.069385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.101708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8907a5-72f3-4945-a7b6-69be3f45c177 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.133076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.147647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.149012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.149472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.150088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] PATH_INFO: `/v3/users/a720729a2a2f4ef8910efeeec32d237b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.152328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] RBAC: Authorizing `identity:get_user(user_id=a720729a2a2f4ef8910efeeec32d237b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:55.154030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:55.156222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 140/286] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:54 2022] GET /identity/v3/users/a720729a2a2f4ef8910efeeec32d237b => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:55.162653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.207331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e554d32-95f9-462d-9812-73439c5b0cc7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.274759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.275479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.275760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.276035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.299691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:55.309788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:55.310683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:55.311896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 147/287] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:55.318469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.335445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325885c-12f1-443d-b6b0-4f5bab7f6e13 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.345682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.402321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.403494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.403956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.404404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.409757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:55.411410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:55.413490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 141/288] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:55 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:55.419867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.475170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f199b8-e268-4cd4-8054-3b4aa25665fa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.546541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.546743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.548102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.548215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.548435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.553063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:55.553063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:55.553063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:55.567853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 148/289] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:55.578336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.602148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff738850-9ad3-4c29-ade9-4008a8323dc5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.638652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.639888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.640370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.641459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/a720729a2a2f4ef8910efeeec32d237b/roles/897f530f43b540ba9dc83bf620bc4e97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.643907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, user_id=a720729a2a2f4ef8910efeeec32d237b, role_id=897f530f43b540ba9dc83bf620bc4e97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:55.645609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:55.683231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.722842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 142/290] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:29:55 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/users/a720729a2a2f4ef8910efeeec32d237b/roles/897f530f43b540ba9dc83bf620bc4e97 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:29:55.724403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b97f-2ef2-4cb1-8d32-26adcff80b08 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.778102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.779385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.779878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.780317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] PATH_INFO: `/v3/users/49602335126a4413adefdb964e9d6728` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:55.782500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] RBAC: Authorizing `identity:get_user(user_id=49602335126a4413adefdb964e9d6728)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:55.785634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:55.786554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 149/291] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:55 2022] GET /identity/v3/users/49602335126a4413adefdb964e9d6728 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:55.792261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:55.955377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add32d59-4fae-4f73-b333-c982733ce786 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:55.978991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:55.994194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:55.994847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:55.995124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:55.995396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:56.074523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:56.076006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:56.223659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 143/292] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:56.229190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:56.360931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4b008-ec18-4c6f-8887-abe0426c9867 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:56.396294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:56.397559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:56.398064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:56.398537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:56.399927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:56.401599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87ce40-b30f-4032-964e-d3ff23c3734d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:56.403836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 150/293] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:56 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:56.409952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:56.453233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:56.574378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9646e-f08e-4be8-a655-b54e86d13123 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:56.620805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:56.621493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:56.621771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:56.622045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:56.622872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=49602335126a4413adefdb964e9d6728) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:56.623186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:56.624613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdf927-99a1-4a58-b444-48c86bbdfd25 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:56.633582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 144/294] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:56 2022] GET /identity/v3/role_assignments?user.id=49602335126a4413adefdb964e9d6728&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:57.180974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910617c-0be8-463b-be38-d458773ec9f0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.181814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910617c-0be8-463b-be38-d458773ec9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.182274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910617c-0be8-463b-be38-d458773ec9f0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.183835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 151/295] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:57.189816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435605e-d2f2-4a2d-9319-e523324e8ac4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.190271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435605e-d2f2-4a2d-9319-e523324e8ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.190370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435605e-d2f2-4a2d-9319-e523324e8ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.211159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8435605e-d2f2-4a2d-9319-e523324e8ac4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:57.212342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8435605e-d2f2-4a2d-9319-e523324e8ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:57.229856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557633d-300e-4a75-aca4-d2547fc50749 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.230483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557633d-300e-4a75-aca4-d2547fc50749 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.230978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557633d-300e-4a75-aca4-d2547fc50749 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.232805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 152/296] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:57.238905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.239453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.239933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.259940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:57.261296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:57.263567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.303303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 145/297] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:29:57.317490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fde083-2a10-4f8a-8732-73bc2dc08b84 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.318182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fde083-2a10-4f8a-8732-73bc2dc08b84 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.318697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fde083-2a10-4f8a-8732-73bc2dc08b84 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.320392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 146/298] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:29:57.326353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.327004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.327287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.343029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 153/299] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:57.343853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.347426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:57.348622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:57.358991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.419832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 147/300] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:57.425939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.454376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762d28-c19a-471f-9555-7d325cdf658b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.490513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.491515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.491935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.492390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.500606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.505368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:57.514192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:57.515304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:57.516302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 154/301] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:57.516982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.539517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff859af8-166a-4fcc-b258-1b8baeac380a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.575557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.576344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.576622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.576897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.578479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:57.580255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:57.582130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 148/302] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:57.587246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.604279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.624552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd26da-b3f7-41b2-bbc5-af3574ff8e2a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.660930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.662047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.662332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.662613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.673492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:57.682885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:57.684538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 155/303] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:29:57.685363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.700484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e64f1-cca2-4abf-955b-85fed2f35cb8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.736415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.736997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.737309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.737583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.738337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:57.738634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:57.740008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:57.748670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 149/304] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:57.749064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.749526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.796782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48669-5cb5-4e8c-880a-df7287766dcf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.832867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.833561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.833833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.834103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] PATH_INFO: `/v3/groups` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.834789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:57.835677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:57.836244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:57.859131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7ab6-bda1-4a62-b313-a564215b3020 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:57.894176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:57.894844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:57.895125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:57.895402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:57.896146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:57.896453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:57.897868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:57.905926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 150/305] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:57.907540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:57.940372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 156/306] 10.222.0.47 () {60 vars in 1134 bytes} [Sat Jul 16 12:29:57 2022] POST /identity/v3/groups => generated 229 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:29:57.950348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.021515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fe74-b7d5-47d8-9150-07428d4dd28f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.061728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.063146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.063667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.064271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] PATH_INFO: `/v3/users/302cf65c37514067be855b6b45fccde3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.069310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:58.082225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] RBAC: Authorizing `identity:get_user(user_id=302cf65c37514067be855b6b45fccde3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.084484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.086850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 151/307] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/users/302cf65c37514067be855b6b45fccde3 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:58.093397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.110912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc0cd4-8e11-4fb4-9e32-79eefd694e92 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.135021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:58.155989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.158097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.158393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.158673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] PATH_INFO: `/v3/users/739d41fecbf44132aade8da48d968b22` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.159960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] RBAC: Authorizing `identity:get_user(user_id=739d41fecbf44132aade8da48d968b22)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.162026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.163696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 157/308] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:57 2022] GET /identity/v3/users/739d41fecbf44132aade8da48d968b22 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:58.170216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.234629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eb94b-1621-41fb-bcc4-acef6938321a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.275421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.275985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.276269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.276544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.277421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, user.id=302cf65c37514067be855b6b45fccde3, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:58.277737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.279195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c994ca7-3332-4252-88da-4fba1f7900f7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.289093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 152/309] 10.222.0.47 () {56 vars in 1308 bytes} [Sat Jul 16 12:29:58 2022] GET /identity/v3/role_assignments?user.id=302cf65c37514067be855b6b45fccde3&scope.system=all&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:58.289093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134ec4-26d8-48fc-a7fa-82699f562673 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.289093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134ec4-26d8-48fc-a7fa-82699f562673 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.289093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134ec4-26d8-48fc-a7fa-82699f562673 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.289093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 153/310] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:29:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:29:58.293607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872988-987f-41ed-9f05-36937e60d435 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.293895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872988-987f-41ed-9f05-36937e60d435 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.294176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872988-987f-41ed-9f05-36937e60d435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.305254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d81045-160a-470e-9f61-622ffae74ffe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.313688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-52872988-987f-41ed-9f05-36937e60d435 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:29:58.315219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-52872988-987f-41ed-9f05-36937e60d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:29:58.352424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.353804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.354309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.354797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] PATH_INFO: `/v3/system/users/739d41fecbf44132aade8da48d968b22/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.357935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=739d41fecbf44132aade8da48d968b22, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.362514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.380980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:58.395624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 154/311] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:29:58.418679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52872988-987f-41ed-9f05-36937e60d435 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.455720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 158/312] 10.222.0.47 () {58 vars in 1334 bytes} [Sat Jul 16 12:29:58 2022] PUT /identity/v3/system/users/739d41fecbf44132aade8da48d968b22/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:29:58.535272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52872988-987f-41ed-9f05-36937e60d435 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.575856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.576679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.577078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.577362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.590375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:29:58.599257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:29:58.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:29:58.601214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 155/313] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:29:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:29:58.606998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.675010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:58.728817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c264-1f86-47e6-a0c1-f65fa06f9baf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.768299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.769074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.769364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.769633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.770433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:58.770735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.772154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.781550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 159/314] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:29:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:58.787125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:58.837106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:58.892897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8b730-0fdd-4bba-8143-306a759cd61e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:58.928145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:58.929022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:58.929326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:58.929599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] PATH_INFO: `/v3/users/a720729a2a2f4ef8910efeeec32d237b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:58.930810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=a720729a2a2f4ef8910efeeec32d237b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:58.932257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:58.934210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 156/315] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:29:58 2022] GET /identity/v3/users/a720729a2a2f4ef8910efeeec32d237b => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:29:58.940163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:59.046742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2efd9-259a-4629-9dc2-142017a67189 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:59.059929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:59.081404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:59.082132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:59.082414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:59.093187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:59.152600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:59.154095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:59.259021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 160/316] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:29:59.264281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:59.368495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4314a-81bc-479f-96fe-4c320d77d2a7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:59.385214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:59.403322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:59.404172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:59.410741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:59.410741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:59.410741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:59.410741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701e340-dd8e-4a59-bfe6-100b9bd19429 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:59.410741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 157/317] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:29:59 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:29:59.415156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:29:59.429696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:29:59.518687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301a71f-9842-49f3-8ba6-d450ca04b525 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:29:59.554501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:29:59.555156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:29:59.555431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:29:59.555701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:29:59.556530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817, user.id=a720729a2a2f4ef8910efeeec32d237b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:29:59.556860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:29:59.558297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffee48-09fd-409e-831a-37806685a4da None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:29:59.565157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 161/318] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:29:59 2022] GET /identity/v3/role_assignments?user.id=a720729a2a2f4ef8910efeeec32d237b&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:00.108766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d777-d5c4-4803-adf0-a1e146e3b1bb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.109780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d777-d5c4-4803-adf0-a1e146e3b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.109965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d777-d5c4-4803-adf0-a1e146e3b1bb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.111845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 158/319] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:00.117234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.117602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.117952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.135081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:00.136273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:00.213075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 162/320] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:00.233955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5d777-d5c4-4803-adf0-a1e146e3b1bb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.335426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5d777-d5c4-4803-adf0-a1e146e3b1bb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:00.369626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:00.370391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.370677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.370952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.372109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:00.373603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70706c39-8852-4d4f-99ee-8de763297a3e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:00.376099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 159/321] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:00.381419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.389877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:00.445082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b42f9c-692f-4460-b0e3-3593f82bdd14 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.445433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b42f9c-692f-4460-b0e3-3593f82bdd14 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.445653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b42f9c-692f-4460-b0e3-3593f82bdd14 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.447031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 160/322] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:30:00.451886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494fab-29fb-464a-ac5e-72837f678ec0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.452166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494fab-29fb-464a-ac5e-72837f678ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.452448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494fab-29fb-464a-ac5e-72837f678ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.469849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0c494fab-29fb-464a-ac5e-72837f678ec0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:00.471299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0c494fab-29fb-464a-ac5e-72837f678ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:00.486394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dc325-21c3-4c7c-b4f4-31642b44ff1c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:00.523547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:00.524006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:00.525110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.525110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.525304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] PATH_INFO: `/v3/groups` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.525916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:00.527381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0937c4-2038-4603-831a-c820bb9bf251 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:00.543779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 161/323] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:00.561605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53509ec-8648-4afe-9803-238471f19b19 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.561892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53509ec-8648-4afe-9803-238471f19b19 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.562314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53509ec-8648-4afe-9803-238471f19b19 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.564171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 162/324] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:30:00.568632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53509ec-8648-4afe-9803-238471f19b19 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.643932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 163/325] 10.222.0.47 () {60 vars in 1134 bytes} [Sat Jul 16 12:30:00 2022] POST /identity/v3/groups => generated 226 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:00.643932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.643932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.643932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.657790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:00.659260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:00.683256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53509ec-8648-4afe-9803-238471f19b19 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:00.721985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:00.722797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.723078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.723351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.728298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:00.736304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:00.737020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 164/326] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:00.744351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:00.745605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:00.746409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 163/327] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:00.752355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.760456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.810129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:00.866710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd2f8-86fd-4a31-b1f8-ef62d9f9c24e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:00.878615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cd63-775f-425d-8442-ec4c93218ea6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:00.912565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:00.913535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.913952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.914230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.915116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:00.915422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:00.916960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:00.928469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:00.928662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 165/328] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:00.929182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:00.929472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:00.929750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:00.933753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:00.944210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:00.951917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:00.952951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:00.954059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:00.954664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 164/329] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:00.960604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:01.007761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:01.060011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4129a3-a990-4af8-9b78-21e213c62fc1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:01.086593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a7a9-bf4a-49d4-8dfc-73d48019ac7c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:01.088161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:01.103578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:01.104463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:01.104747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:01.105023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] PATH_INFO: `/v3/users/302cf65c37514067be855b6b45fccde3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:01.107034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] RBAC: Authorizing `identity:get_user(user_id=302cf65c37514067be855b6b45fccde3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:01.108591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:01.111157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 166/330] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/users/302cf65c37514067be855b6b45fccde3 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:01.118006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:01.128523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:01.129302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:01.129631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:01.129930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:01.130668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:01.130975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:01.132342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:01.144749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 165/331] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:01.148649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:01.230438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4156e0-fadb-4391-b98c-d35c0e4f7cec None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:01.258104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3b122-2392-497e-99e3-31421bf7c31b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:01.265991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:01.269511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:01.270583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:01.271007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:01.271420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] PATH_INFO: `/v3/system/users/302cf65c37514067be855b6b45fccde3/roles/897f530f43b540ba9dc83bf620bc4e97` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:01.273187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=302cf65c37514067be855b6b45fccde3, role_id=897f530f43b540ba9dc83bf620bc4e97)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:01.274826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd1621-2244-4f99-afb0-42278bbd5caf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:01.301154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:01.302028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:01.302304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:01.302788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] PATH_INFO: `/v3/users/739d41fecbf44132aade8da48d968b22` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:01.303890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] RBAC: Authorizing `identity:get_user(user_id=739d41fecbf44132aade8da48d968b22)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:01.305390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:01.307229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 166/332] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:30:01 2022] GET /identity/v3/users/739d41fecbf44132aade8da48d968b22 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:01.313562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:01.340101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 167/333] 10.222.0.47 () {58 vars in 1334 bytes} [Sat Jul 16 12:30:01 2022] PUT /identity/v3/system/users/302cf65c37514067be855b6b45fccde3/roles/897f530f43b540ba9dc83bf620bc4e97 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:01.352098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:01.444250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22085848-99d0-45cb-a42c-c42e24608147 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:01.486236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:01.486901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:01.487217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:01.487493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:01.488344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] RBAC: Adding query filter params (role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, user.id=739d41fecbf44132aade8da48d968b22, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:01.488654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:01.490201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d7b1-5fc7-4017-ad48-60abc22870b4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:01.505850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 167/334] 10.222.0.47 () {56 vars in 1308 bytes} [Sat Jul 16 12:30:01 2022] GET /identity/v3/role_assignments?user.id=739d41fecbf44132aade8da48d968b22&scope.system=all&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:03.198122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb708b9-8517-4fb0-9585-7b2adccb0a33 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.198668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb708b9-8517-4fb0-9585-7b2adccb0a33 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.198668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb708b9-8517-4fb0-9585-7b2adccb0a33 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.200251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 168/335] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:03.219604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.220534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.220534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.246270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:03.247462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:03.329382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 168/336] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:03.350419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb708b9-8517-4fb0-9585-7b2adccb0a33 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:03.464941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb708b9-8517-4fb0-9585-7b2adccb0a33 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:03.501837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:03.502641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.502919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.503191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.513604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-cacd1ca3-817c-4eeb-aefa-2c6d47817814 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:03.521496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:03.522409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:03.523224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 169/337] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:03.529918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:03.560811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:03.636902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c7e-a8be-4082-adfd-693550a2c5c7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.636902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c7e-a8be-4082-adfd-693550a2c5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.636902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736c7e-a8be-4082-adfd-693550a2c5c7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.637686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 170/338] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:03.643164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.643707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.644213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.670512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc9063-f5c8-4907-9578-0b6eb5834be7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:03.686081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:03.688084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:03.710218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:03.711230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.711728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.712169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:03.713089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:03.713569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:03.715199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:03.725881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 169/339] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:03.732315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:03.799602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:03.831906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 171/340] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:03.841020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:03.846829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daacde-b04d-418d-b5ee-71f3c8710236 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:03.878363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:03.883302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:03.884208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:03.884662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:03.885149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.015193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.017187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:04.102019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1fb19-43ee-4e76-9715-06add6f603cd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.138485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:04.160911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 170/341] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:30:04.166817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.193759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.194605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.194894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.195177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.225982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:04.240477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:04.241442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:04.242239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 172/342] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:04.254000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.272814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0485f-6b1b-4f7b-a8af-0f0c322f9682 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.310650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.311513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.311797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.312743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.313955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.315646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:04.318040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 171/343] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:04.323640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.340534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:04.389444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:04.426384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a1426-6ad4-4fc2-8fc2-f6014584f46e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.439465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a88ec2-6e40-450f-a48f-b3865eec8b1b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.475623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.476814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.477272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.477778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.485853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.486308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.486788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.487079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.487356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.488125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:04.488432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.488820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:04.489838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:04.491339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 172/344] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:04.497333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.503414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 173/345] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:04.508775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.613376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e6123-dab7-4b81-b6c2-830f13f418ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.628339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:04.651093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.651751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.652028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.652303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.653174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=fa56928f7a294120ac8fe81e41c3a013) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:04.653485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.654924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0760ea-22d2-4042-a47b-952190f94b87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:04.655119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f69-51ab-4638-8332-2b89eb3fbd5a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:04.656679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:04.662699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 173/346] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:04.664039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a212-916b-472c-86aa-1c572973f3af None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.664327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a212-916b-472c-86aa-1c572973f3af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.664598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a212-916b-472c-86aa-1c572973f3af None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.669538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 174/347] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:04.676692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.679463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.679463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.701097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:04.701334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:04.701754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:04.702041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:04.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:04.702419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:04.777086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 175/348] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:04.813845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:04.839083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:04.840594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e02ae-d1cd-481e-8ac5-4c0921a73488 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.001622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011f5f-4ba3-4500-be11-0ce54a22584e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.012979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 174/349] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:30:05.014645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fd4f-277a-4136-84db-9d4a022907e7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.015685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fd4f-277a-4136-84db-9d4a022907e7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.016174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fd4f-277a-4136-84db-9d4a022907e7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.017479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.018227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 175/350] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:05.025955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fd4f-277a-4136-84db-9d4a022907e7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.031855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.073684 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.074798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.074909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.075213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.109149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fb26328b-5767-4607-8b97-0cdb082e8639 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:05.127013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:05.127992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 12:30:05.128825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:05.130980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 176/351] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:05.134251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.136791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.137255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.166467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:05.168130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:05.185738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fd4f-277a-4136-84db-9d4a022907e7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.261902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 177/352] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:05.262313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.262966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.263890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.264322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.264739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.269800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.271425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.273661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 176/353] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:05.279225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.383928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.439482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcd908-8a79-4ccc-af76-af85793c22eb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.442211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.446840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ffc0-63b2-4d6c-869c-724306b2e6fb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.478906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.479494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.479775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.480048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.481581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.483243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.484869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 177/354] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:05.489713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.495570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.496510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.496972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.497507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.498452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:05.498956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.501151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9c514-fa35-46d0-a1e1-bf24ab7d376b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.511020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 178/355] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:05.513917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3896a-c205-49d3-a3fd-a099a7276e16 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.514432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3896a-c205-49d3-a3fd-a099a7276e16 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.515019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3896a-c205-49d3-a3fd-a099a7276e16 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.517130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 179/356] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:05.518056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3896a-c205-49d3-a3fd-a099a7276e16 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.611796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef660-a293-4664-8262-3e758f29d537 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.633655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3896a-c205-49d3-a3fd-a099a7276e16 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.649372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.650148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.650431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.650706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.659563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.664339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:05.673895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:05.675033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:05.676021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:05.677262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 178/357] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:05.678132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.679041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.679961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.680468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.680919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.682128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=fa56928f7a294120ac8fe81e41c3a013) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:05.689003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.689003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9263a0f-39be-4b81-9c30-aa581401481f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 180/358] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=99bae25deed04db6ab077873f28587b2 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:05.692938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.693495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.694062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.712498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:05.714033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:05.737513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.792446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 181/359] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:05.793690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.842851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d6b8f-1a15-4365-8c48-259973d12bcd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.882149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.883041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.883322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.883597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] PATH_INFO: `/v3/users/302cf65c37514067be855b6b45fccde3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.885245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] RBAC: Authorizing `identity:get_user(user_id=302cf65c37514067be855b6b45fccde3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.886798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86475e6-0bac-4c6c-94f9-b2218e35d8e0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.888736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 179/360] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/users/302cf65c37514067be855b6b45fccde3 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:05.890572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68a78-0056-48be-ba30-c31ba1300958 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.891016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68a78-0056-48be-ba30-c31ba1300958 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.891642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68a78-0056-48be-ba30-c31ba1300958 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.897382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 180/361] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:05.898280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68a78-0056-48be-ba30-c31ba1300958 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.915693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54aeb-8e5f-4859-88ee-6eb2bb4881d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:05.957258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:05.958377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:05.958859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:05.959343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:05.960451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:05.961004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:05.962696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:05.973335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 182/362] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:05.974557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:05.979291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:05.994787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.021278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68a78-0056-48be-ba30-c31ba1300958 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.061728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.063185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.063465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.063733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.075849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-dfaa57c9-55f3-4a16-919b-bdbdcb982f51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:06.086160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:06.087107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:06.088316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 181/363] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:06.089259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.089259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.089553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.099600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5a42-4066-4810-87b9-52616db6621a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.114449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:06.115697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:06.185540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.186315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.186594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.187444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.188347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] RBAC: Adding query filter params (role.id=897f530f43b540ba9dc83bf620bc4e97, user.id=302cf65c37514067be855b6b45fccde3, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:06.188664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.190182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:06.206016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 183/364] 10.222.0.47 () {56 vars in 1308 bytes} [Sat Jul 16 12:30:05 2022] GET /identity/v3/role_assignments?user.id=302cf65c37514067be855b6b45fccde3&scope.system=all&role.id=897f530f43b540ba9dc83bf620bc4e97 => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:06.206942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.208304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 182/365] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:06.209205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.282980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.325789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.337638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e15f8-4e2a-49e8-8e78-cfe30100604e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.356110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f8c45-c3d5-41b4-a2c7-22003012b3f2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.395106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.395769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.396053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.396331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.431609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.432348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.432636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.432910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.433687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:06.433988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.435377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:06.453373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 183/366] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:06.456809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.498868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.501056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:06.600637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c2415-962d-4387-939f-f57a48dfedbf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.611181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.615569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 184/367] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:06.616416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.622557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.642979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.643796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.644111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.644392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.656654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:06.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:06.666750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 184/368] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:06.668684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.727210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be41f-622b-4957-9f5e-6db5220e8d36 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.766910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.767674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.768011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.768421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.776945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce470-8786-4a72-8e8b-b0120e916cb4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.823552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.824483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.824767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.825051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.826262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.828534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:06.830545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 185/369] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:06.831148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.837423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.862474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.864274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:06.884051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:06.944242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c277783-1d24-499e-a91e-94e25952d901 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:06.979258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 185/370] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:06.980444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:06.990693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:06.992050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:06.992562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:06.993357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:06.994615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:06.995140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:06.998524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.010005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 186/371] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.011207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.074569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.102997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4090c-a914-4542-b4ea-e851c06a6406 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.120994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aa032-396d-4dfe-9540-3a53c111cae0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.130225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.144069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.145296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.145782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.146252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.147637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.149379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.151285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 186/372] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:06 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.152390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.161725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.163121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.163965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.164622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.165686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.167220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.175294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 187/373] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.183409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.273917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad896d-05ee-42cd-b8ae-89cdf001b92f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.291538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc924c0-7f50-48f5-93bb-9e6b5e378bfe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.318123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.319042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.319467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.319890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.334929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.335105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.335781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.336081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.336479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.340631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:07.340631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.340631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.348613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 188/374] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.348613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.410059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.447961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.449491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.491492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10a9a-0ec8-4274-acd6-65c705648588 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.534924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.536133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.536621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.537240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.539501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.541188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d972c22-206b-4547-85da-eacb32de6808 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.543080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 189/375] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.544565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346017f-cadf-4729-9978-bf7a50cea874 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.545079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346017f-cadf-4729-9978-bf7a50cea874 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.545577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346017f-cadf-4729-9978-bf7a50cea874 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.548117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 190/376] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:07.553650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.554190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.554671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.574038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:07.575989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:07.576981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 187/377] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:07.581906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.633416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.661089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 191/378] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:07.662268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.700129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1257645-97a1-49f3-830c-86de3fe13d96 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.728498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.743514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.744592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.744890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.745195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.746031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:07.746468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.748212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.756005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 188/379] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=99bae25deed04db6ab077873f28587b2 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.756883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.810952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189d33-2a42-42b5-ab74-f26404519f3c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.856942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.858242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.859356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.859855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] PATH_INFO: `/v3/projects/5d986eebab5a45d29f4ae51c867a19af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.861628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] RBAC: Authorizing `identity:get_project(project_id=5d986eebab5a45d29f4ae51c867a19af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:07.863433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:07.865590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 192/380] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/projects/5d986eebab5a45d29f4ae51c867a19af => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:07.872632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:07.895789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eb3f8-3f31-4d0a-ad3d-330371a9fcb0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:07.909430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:07.964004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:07.964734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:07.965025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:07.965348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:07.989663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:08.001770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:08.002761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:08.003554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:08.008064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 189/381] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:08.013957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.084979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b4a2-659c-4314-b028-ed7475f852ba None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.101491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:08.161529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.162877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.163359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.169084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] PATH_INFO: `/v3/groups/dbb9ec530c6945c08b713522ca1aac8f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.169226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a63b-b990-42a5-84b0-07fe306a4934 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.191768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] RBAC: Authorizing `identity:get_group(group_id=dbb9ec530c6945c08b713522ca1aac8f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.197842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:08.199996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 193/382] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:07 2022] GET /identity/v3/groups/dbb9ec530c6945c08b713522ca1aac8f => generated 229 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:08.207505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.223977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.224582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.224867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.225177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.225942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:08.226252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.227641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:08.239985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 190/383] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:08.246129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.305454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:08.393448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c803f-cf80-45ea-b5c0-9b7a753a1a67 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.447522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f20b6-1a15-4aca-9e2b-28907fa9defc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.449477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.450575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.450980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.451359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.452686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.454278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903b6c2-cc17-40e7-aeb8-452bc854b352 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:08.456470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 191/384] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:08 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:08.457819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b249f-31da-4983-89b8-1d19267cd0aa None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.458206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b249f-31da-4983-89b8-1d19267cd0aa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.458583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b249f-31da-4983-89b8-1d19267cd0aa None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.459934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 192/385] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:08.464805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b249f-31da-4983-89b8-1d19267cd0aa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.491405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.491994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.492276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.492548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.493421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] RBAC: Adding query filter params (group.id=dbb9ec530c6945c08b713522ca1aac8f, role.id=a495d38f67e447309647ac782785bd87, scope.project.id=5d986eebab5a45d29f4ae51c867a19af) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:08.493732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.495352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93369ee-89f4-4b06-ae4f-2ec9f2cbeca5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:08.503033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 194/386] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:08 2022] GET /identity/v3/role_assignments?group.id=dbb9ec530c6945c08b713522ca1aac8f&scope.project.id=5d986eebab5a45d29f4ae51c867a19af&role.id=a495d38f67e447309647ac782785bd87 => generated 260 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:08.504617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.504904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.505204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.524018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:08.525256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:08.564019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:08.598603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:08.611390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b249f-31da-4983-89b8-1d19267cd0aa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.614944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 195/387] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:08.658696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.745905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.747145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.747630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.748124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.754351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.755367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:08.759363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 193/388] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:08 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:08.767207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:08.919036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a03472-4547-468e-a951-30258b86e57e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.950195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54d7cb-1e40-4d74-a38a-4b2b97d7b169 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:08.966972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:08.967914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:08.968865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:08.968865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:08.972467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, group_id=56fe88cd567b4399b4dbde7a8d6af2d9, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:08.972716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:09.015009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:09.015813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:09.016147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:09.016432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:09.037027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:09.047044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:09.048000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:09.048816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 196/389] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:09.055370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:09.076764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 194/390] 10.222.0.47 () {58 vars in 1442 bytes} [Sat Jul 16 12:30:08 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:09.118708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:09.276845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647242a4-9f27-4ef5-a220-e24462c0d371 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:09.317633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:09.318314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:09.318608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:09.318894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:09.319652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:09.319966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:09.321370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:09.330153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 197/391] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:09.335935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:09.386904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:09.503865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998d61a-d86c-4951-a998-8ec1d808aebb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:09.609871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:09.611377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:09.611377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:09.611377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:09.612521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:09.613990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:09.615929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 195/392] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:09 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:09.622980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:09.765546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:09.793161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a66b6-70b6-4eef-91a7-10a5a234f68a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:09.861831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:09.862631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:09.862917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:09.863193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:09.864426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:09.865950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3dfa5-04b4-4578-8cfc-4e404000d1a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:09.868010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 198/393] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:09 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:09.874579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:10.083765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:10.113426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a157f7-ccce-490e-b716-316d91e04821 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.114038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a157f7-ccce-490e-b716-316d91e04821 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.114508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a157f7-ccce-490e-b716-316d91e04821 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.116343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 199/394] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:10.119877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574775c1-52d7-4da2-a17f-2625d59c2011 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:10.126678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.127246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.132406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.174156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:10.175393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:10.200442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:10.201481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.201756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.202022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.206405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa56928f7a294120ac8fe81e41c3a013, group_id=56fe88cd567b4399b4dbde7a8d6af2d9, role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:10.207852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:10.278537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 200/395] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:10.291949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 196/396] 10.222.0.47 () {58 vars in 1442 bytes} [Sat Jul 16 12:30:09 2022] PUT /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/99bae25deed04db6ab077873f28587b2 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:10.306618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:10.404917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:10.423380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a72c4-881a-4454-a9a1-a6196b172a44 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:10.464587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:10.465464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.465747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.466020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.478303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:10.486616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:10.487693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:10.488543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 201/397] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:10.493822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:10.616087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122be51-5031-4f2d-8a13-fe09831dd9e1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:10.660244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:10.660849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.661161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.661436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.662189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:10.662492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:10.664617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab440be3-ba3e-4ca4-94bc-958442e91930 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:10.677682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 197/398] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:10.684843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:10.878594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66bef-37db-4e99-b0d2-eae3c22660f0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.879185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66bef-37db-4e99-b0d2-eae3c22660f0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.879648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66bef-37db-4e99-b0d2-eae3c22660f0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.881196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 198/399] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:10.885699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7b32-cb7a-4b70-b487-d0edfd0fdcae None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:10.887976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.888266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.888546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.920488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:10.922547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:10.949386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:10.957439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:10.963580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:10.963580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:10.963580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:10.963580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:10.963580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:10.964525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 202/400] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:10 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:10.971658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.004339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.056756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 199/401] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:11.096559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.211480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b7ff-80c1-4286-b80a-e0b4a5ea64da None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.218201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8f002-beb4-4aff-9387-ee7396ff36b0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.255197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.256033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.256320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.256596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.257838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:11.259268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cd069-c351-4bbd-b05e-a9d0d0a3a781 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:11.260047 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.261098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.261364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 203/402] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:10 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:11.261814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.262293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.262695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11c1bc-14db-42aa-91ac-c09b31d9ffe2 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.262984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11c1bc-14db-42aa-91ac-c09b31d9ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.263261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11c1bc-14db-42aa-91ac-c09b31d9ffe2 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.268495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 204/403] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:11.269470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11c1bc-14db-42aa-91ac-c09b31d9ffe2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.286446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b4c2b515-b96c-4a12-a010-4dd95d663c9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:11.295527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:11.298693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:11.299515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:11.302359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 200/404] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:11.303867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.304373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.304859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.311976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.325644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:11.328292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:11.381115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.411206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11c1bc-14db-42aa-91ac-c09b31d9ffe2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.417192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 201/405] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:11.418068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.480476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.481739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.482225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.482718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.485611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, group_id=56fe88cd567b4399b4dbde7a8d6af2d9, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:11.487286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:11.540109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec33f8-c2bf-42bd-bec6-99fea65947d9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.581898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 205/406] 10.222.0.47 () {58 vars in 1442 bytes} [Sat Jul 16 12:30:11 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:11.583176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.600914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.602075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.602626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.603179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.604981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:11.606274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:11.609026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:11.613927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.630853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 202/407] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:11.636932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.779995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e2e0-5c7d-4dd7-a4dd-0ea107b2fd4c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.818072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d747a-bea1-44fd-8591-88232c22c1de None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:11.823515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.824314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.824591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.829063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.839037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.843594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0d91ca87-db0b-45d4-bfef-e4506852bfe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:11.851965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:11.855887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:11.856889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 206/408] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:11.858072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54340-da31-4cdc-8f8f-db215df1f2d3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.859188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54340-da31-4cdc-8f8f-db215df1f2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.859685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54340-da31-4cdc-8f8f-db215df1f2d3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.861486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 207/409] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:11.866247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b54340-da31-4cdc-8f8f-db215df1f2d3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:11.866462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:11.866462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.866462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.866688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.867901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:11.869438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cebf6-a323-4fa1-bdc5-16a5438b5599 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:11.877374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 203/410] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:11.879280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:11.879573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:11.879867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:11.905913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:11.907422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:11.976262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:11.986294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 204/411] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:11.987532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.003039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b54340-da31-4cdc-8f8f-db215df1f2d3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.052577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.053204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.053489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.053766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.055067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:12.055383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.057055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.069847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 208/412] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:12.070844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.117190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591f4a3-e4fd-4c71-ac9a-f90a749c8fbf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.159322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.163718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.164586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.164871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.165187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.167015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.176608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.178865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 205/413] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:11 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:12.181014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.224247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac238758-c6df-4967-a8a7-d00199497111 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.256310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.271000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.271802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.272082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.272350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.284117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:12.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:12.293483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:12.294297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:12.298629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 209/414] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:12.299456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.308097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8028d-c356-4d9a-aaf4-6457bd5f5df7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.356218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.357645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.358956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.359432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] PATH_INFO: `/v3/groups/dbb9ec530c6945c08b713522ca1aac8f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.361253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] RBAC: Authorizing `identity:get_group(group_id=dbb9ec530c6945c08b713522ca1aac8f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.363462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.365625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 206/415] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:12 2022] GET /identity/v3/groups/dbb9ec530c6945c08b713522ca1aac8f => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:12.370532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.434316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74f77-d6eb-4cff-ab37-c722122c5cbd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.466058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.482904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.484484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703b41-477d-41cd-81ba-906b9261a754 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.487635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.487635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.487635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.489238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817, group_id=56fe88cd567b4399b4dbde7a8d6af2d9, role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.490913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.515693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.521794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.533367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.533772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 207/416] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:12.538397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.595607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 210/417] 10.222.0.47 () {58 vars in 1442 bytes} [Sat Jul 16 12:30:12 2022] PUT /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817/groups/56fe88cd567b4399b4dbde7a8d6af2d9/roles/99bae25deed04db6ab077873f28587b2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:12.596874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.706006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94781aec-11f7-47ed-b392-02addb6ace6a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.788715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.789618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.793087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.793087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] PATH_INFO: `/v3/projects/5d986eebab5a45d29f4ae51c867a19af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.793087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] RBAC: Authorizing `identity:get_project(project_id=5d986eebab5a45d29f4ae51c867a19af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.796135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.800182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 208/418] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:12 2022] GET /identity/v3/projects/5d986eebab5a45d29f4ae51c867a19af => generated 344 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:12.805935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:12.826645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276a142-10d8-4102-8a4b-5811a483f8d8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.865076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.865756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.866038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.866313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.899785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.918734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a12e94-7704-43d8-9058-cd5630b14b10 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:12.919682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:12.940105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.941877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:12.953553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:12.954241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:12.954522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:12.954792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] PATH_INFO: `/v3/projects/5d986eebab5a45d29f4ae51c867a19af/groups/dbb9ec530c6945c08b713522ca1aac8f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:12.956970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d986eebab5a45d29f4ae51c867a19af, group_id=dbb9ec530c6945c08b713522ca1aac8f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:12.962517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d06fe-17a5-423b-9bba-0fec89ca0e87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:13.041307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 209/419] 10.222.0.47 () {58 vars in 1442 bytes} [Sat Jul 16 12:30:12 2022] PUT /identity/v3/projects/5d986eebab5a45d29f4ae51c867a19af/groups/dbb9ec530c6945c08b713522ca1aac8f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:13.045413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4150-edd7-4203-9900-478ec0f266a1 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:13.046634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4150-edd7-4203-9900-478ec0f266a1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:13.047087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4150-edd7-4203-9900-478ec0f266a1 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:13.052708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 210/420] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:13.056873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:13.057502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:13.057957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:13.071741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 211/421] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:13.077398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:13.096413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:13.100043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:13.235973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:13.260134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591be076-0eeb-4163-9119-4fb7adfab6bc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:13.286152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 211/422] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:13.314995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:13.316223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:13.316695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:13.316881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:13.317398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:13.320204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:13.321902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:13.324137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 212/423] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:13 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:13.334146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:13.426306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:13.555650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3b07f-73da-4572-a22e-eb040ff8328e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:13.607782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5dc1-cef3-4aa8-97cb-73c4d11568ca None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:13.658346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:13.659523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:13.660145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:13.660761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:13.701524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:13.712569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:13.713968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:13.714784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 212/424] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:13.725068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:13.737849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:13.739266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:13.739266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:13.739449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:13.742143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:13.744983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:13.751591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 213/425] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:13 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:13.751591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:13.793435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:13.925068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4adb-1631-48dc-80cd-ac01bbdd1ec7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:13.969071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe1b9e-dc18-4e74-af7b-bae8683bac96 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:13.992085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:14.004499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:14.006357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.009787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.010214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.011184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=fa56928f7a294120ac8fe81e41c3a013) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:14.011660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:14.013260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458573-d4c5-4fab-b6e7-e71fb6d4cb0b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:14.028810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 214/426] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:13 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 611 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:14.031011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691374-577d-4df8-9460-ca1baef17322 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.033572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691374-577d-4df8-9460-ca1baef17322 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.034534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691374-577d-4df8-9460-ca1baef17322 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.037264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 215/427] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:14.047595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:14.049994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.050511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.051004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.052286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:14.052891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:14.054647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:14.058940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.059515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.060007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.121915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 213/428] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:14.128923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:14.133657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:14.143157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:14.309083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 216/429] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:14.341066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:14.375848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:14.440175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe922cb-03d7-461c-8b55-9d3bfd058f19 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:14.513062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:14.513062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.517063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.517063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.541071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047a73a-2736-4f60-b5b2-a817268bae57 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:14.600309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:14.612365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:14.614414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.614707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.614985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.649115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:14.658244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:14.659167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 12:30:14.659967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:14.663413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:14.665732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:14.668007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 217/430] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:14.668007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:14.829142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:14.856709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 214/431] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:14.869069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:14.901069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058f8-8a56-487e-bbf9-2fde89882c67 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:14.949062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:14.949062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.949062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.949062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.952290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:14.952290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:14.954149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d979326-e158-4ccd-9b78-327c01583ca3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:14.977067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 218/432] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:14.977067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee45b-e681-4d3f-8276-d0ed9ceffe11 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:14.977067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee45b-e681-4d3f-8276-d0ed9ceffe11 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:14.977067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee45b-e681-4d3f-8276-d0ed9ceffe11 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:14.979318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 219/433] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:14.985063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee45b-e681-4d3f-8276-d0ed9ceffe11 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:14.987660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.020337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9921f8-2038-44a5-ada0-f1b3afdaf90b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.064180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.065064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.065359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.065622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] PATH_INFO: `/v3/projects/fa56928f7a294120ac8fe81e41c3a013` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.066813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=fa56928f7a294120ac8fe81e41c3a013)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.068245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ce08-b831-43ba-9d29-88d35e419ea9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.074491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 215/434] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:14 2022] GET /identity/v3/projects/fa56928f7a294120ac8fe81e41c3a013 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.074491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.074491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.074491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.090745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:15.091854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:15.177021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee45b-e681-4d3f-8276-d0ed9ceffe11 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.192841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 216/435] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:15.201242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.250337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.251277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.251742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.252015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.254983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.307770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.338956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342ab3-898c-436a-a5f6-2d648aa7bc99 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.378113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.379784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb542a0-05da-4c0f-816f-9dd9488aa4b9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.385494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.386413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.386694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.386986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.388189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.389701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.391629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 217/436] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.392401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.492590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 220/437] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:15.493999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f465a1-b089-48b3-993a-2722a9a610be None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.494288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f465a1-b089-48b3-993a-2722a9a610be None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.494566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f465a1-b089-48b3-993a-2722a9a610be None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.495862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 221/438] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:15.497173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f465a1-b089-48b3-993a-2722a9a610be None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.504443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4f3a4-e7bb-44af-a84c-e3fb05867281 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.532506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.541051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.541881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.542171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.542446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.549731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.554673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:15.563308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:15.564472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 218/439] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:15.565422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.610876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f465a1-b089-48b3-993a-2722a9a610be None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.646353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.647029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.647367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.647718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.648595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=fa56928f7a294120ac8fe81e41c3a013) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:15.648974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.650498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc0533-a3c2-4f78-83c4-6efb5668111e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.666062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 222/440] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=fa56928f7a294120ac8fe81e41c3a013&role.id=99bae25deed04db6ab077873f28587b2 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.667301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.667598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.667857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.679000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289faf82-4b89-4717-a34b-c42bc870412f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.696204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:15.697472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:15.721260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.722151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.722437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.722714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.724487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.726066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.732617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 219/441] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.732617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.793780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 223/442] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:15.795098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.802044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.834507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:15.851208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a85b-151c-4b9f-9445-375ec45f260b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.895656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.896288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.896564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.896757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.897880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:15.898182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.899742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.910495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 220/443] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.911655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:15.923023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d3124-3155-426b-85fc-49e7db8f4173 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:15.970103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:15.971033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:15.971316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:15.971586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:15.972870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:15.974372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:15.976061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 224/444] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:15.977543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.043567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed526545-893a-4490-9df1-6718d43de4be None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.070412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.078841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.079756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.080034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.080309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.085224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.090537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:16.090651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c6ac7-4250-4963-b6eb-bd6d7e728ec0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:16.099483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:16.100566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:16.101608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:16.105255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 221/445] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:16.107922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.126794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.127540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.128163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.128440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.199904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.201371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.207125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958d681-b2a0-4e69-8a96-ebc86298f72b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.240510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.242273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.242273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.242273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.242700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:16.243018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.244437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.250495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 222/446] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:16.251279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.301137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.313510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 225/447] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:16.317093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.337786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.398387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8281-6f20-4e83-9041-427d5ceba00a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.437341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.438139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.438491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.438847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.439595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:16.439894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.439989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c605a-2277-4573-81e6-b0aea9bd3bcd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.441366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 223/448] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:16.458708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.487621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.488476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.488758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.489020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] PATH_INFO: `/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.490413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] RBAC: Authorizing `identity:get_project(project_id=86b52bd9c88e4a5d9a6a87db1ef6b817)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.491634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.495807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 226/449] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/projects/86b52bd9c88e4a5d9a6a87db1ef6b817 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:16.500304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.574176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.585745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6fe28-d4a0-4e41-b410-6838d9305b9e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.601099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.620736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4584cc-ac4a-45b0-bda7-7e8f9b123437 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.625201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.625853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.626131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.626397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.654265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.655340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.655677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.656015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] PATH_INFO: `/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.657222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] RBAC: Authorizing `identity:get_group(group_id=56fe88cd567b4399b4dbde7a8d6af2d9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.658654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.660294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 227/450] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/groups/56fe88cd567b4399b4dbde7a8d6af2d9 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:16.666142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.694031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.695415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.766425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ae173-c522-4f24-b9df-3659b7ceed65 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.792918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 224/451] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:16.794780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.798474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:16.799792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.800444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.800726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.801001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.809080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] RBAC: Adding query filter params (group.id=56fe88cd567b4399b4dbde7a8d6af2d9, role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:16.809080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.809080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.809360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 228/452] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/role_assignments?group.id=56fe88cd567b4399b4dbde7a8d6af2d9&scope.project.id=86b52bd9c88e4a5d9a6a87db1ef6b817&role.id=99bae25deed04db6ab077873f28587b2 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:16.813562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:16.903995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12234c09-43d2-4c3f-8835-469cfe6e3a77 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:16.948659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:16.949518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:16.949804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:16.950074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] PATH_INFO: `/v3/projects/5d986eebab5a45d29f4ae51c867a19af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:16.951303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] RBAC: Authorizing `identity:get_project(project_id=5d986eebab5a45d29f4ae51c867a19af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:16.953799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:16.957079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 225/453] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/projects/5d986eebab5a45d29f4ae51c867a19af => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:16.961174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:17.103220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32887b6-1ab7-4d52-9754-a0e2d95e3267 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:17.143723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:17.144591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:17.144868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:17.145265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] PATH_INFO: `/v3/groups/dbb9ec530c6945c08b713522ca1aac8f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:17.146461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] RBAC: Authorizing `identity:get_group(group_id=dbb9ec530c6945c08b713522ca1aac8f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:17.147908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae4009-5112-4b24-8c18-f6037e3f98c0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:17.149874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 229/454] 10.222.0.47 () {56 vars in 1180 bytes} [Sat Jul 16 12:30:16 2022] GET /identity/v3/groups/dbb9ec530c6945c08b713522ca1aac8f => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:17.155727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:17.250600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:17.267419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa066-9ce7-4840-af61-b32976f02b0a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:17.300751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:17.301439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:17.301724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:17.301997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:17.302825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] RBAC: Adding query filter params (group.id=dbb9ec530c6945c08b713522ca1aac8f, role.id=a495d38f67e447309647ac782785bd87, scope.project.id=5d986eebab5a45d29f4ae51c867a19af) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:17.303138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:17.304565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0f845-c7ef-4cbb-ae5c-8233a766633b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:17.310352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 226/455] 10.222.0.47 () {56 vars in 1376 bytes} [Sat Jul 16 12:30:17 2022] GET /identity/v3/role_assignments?group.id=dbb9ec530c6945c08b713522ca1aac8f&scope.project.id=5d986eebab5a45d29f4ae51c867a19af&role.id=a495d38f67e447309647ac782785bd87 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:19.945075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fef6-9d3f-4a36-880e-2ef891f145a4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:19.945075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fef6-9d3f-4a36-880e-2ef891f145a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:19.945075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fef6-9d3f-4a36-880e-2ef891f145a4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:19.947570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 230/456] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:19.953129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1287ac-4d83-4523-b3ef-f3318ae6467b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:19.953964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1287ac-4d83-4523-b3ef-f3318ae6467b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:19.954437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1287ac-4d83-4523-b3ef-f3318ae6467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:19.982104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6f1287ac-4d83-4523-b3ef-f3318ae6467b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:19.983702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6f1287ac-4d83-4523-b3ef-f3318ae6467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:20.094939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c6b2b-417c-4271-a351-759b3ca0cfa6 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.095282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c6b2b-417c-4271-a351-759b3ca0cfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.095489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c6b2b-417c-4271-a351-759b3ca0cfa6 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.099613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 227/458] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:20.104320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 231/458] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:20.120094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7922e-0c60-40d5-8fc9-d4caa51006d0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.120257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c02af-f046-42ab-8493-fc8985e72508 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.120468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7922e-0c60-40d5-8fc9-d4caa51006d0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.120561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c02af-f046-42ab-8493-fc8985e72508 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.120640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7922e-0c60-40d5-8fc9-d4caa51006d0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.120721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c02af-f046-42ab-8493-fc8985e72508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.122056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 228/459] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:20.125404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7922e-0c60-40d5-8fc9-d4caa51006d0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.141636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b89c02af-f046-42ab-8493-fc8985e72508 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:20.143490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b89c02af-f046-42ab-8493-fc8985e72508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:20.221336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.232212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 232/460] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:20.236239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.236348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.236348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.236348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.263316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:20.264604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:20.265146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7922e-0c60-40d5-8fc9-d4caa51006d0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.310307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.311581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.312112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.312584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.323722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.325495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:20.347398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:20.348308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:20.349176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:20.349176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:20.349176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.349176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:20.349401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 233/461] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:20.350242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.351619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 229/462] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:20.362106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.458475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18acdf5-df6e-4553-85a3-30f3c3493530 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.485102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.489983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a83a-8f36-4e71-bdea-231749925ebb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.495632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.500523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.500523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.500523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.512530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.516087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.520810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.529126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.530389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.530766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.531109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.531912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:20.532341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.533819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:20.538497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:20.547256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:20.548085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:20.548085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:20.548085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.548085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:20.548085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 234/463] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:20.548487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.551630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 230/464] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:20.553081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.669357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27051-d68b-4b75-888d-dda715d841c2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.683602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ab09-c6e5-453b-9103-c9af338b6a56 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.707875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.708667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.708947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.709251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.720049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.720713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.720997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.721190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.722582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.722582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:20.722743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.722812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.724330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.740947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 231/465] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:20.741645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-3b44e6ff-bab9-4fcb-b5b1-ebc534a72f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:20.745866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:20.746708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:20.747562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:20.747562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:20.747562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:20.747562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:20.747562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 235/466] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:20.748943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e30330-3819-468b-be90-e08fe9ffb7f9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.757498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e30330-3819-468b-be90-e08fe9ffb7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.757498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e30330-3819-468b-be90-e08fe9ffb7f9 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.757498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 236/467] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:20.757498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e30330-3819-468b-be90-e08fe9ffb7f9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:20.782263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.794491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:20.850969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2933-2d45-4d08-8d9c-bd5713fe266b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.862831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e30330-3819-468b-be90-e08fe9ffb7f9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:20.886001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.886665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.886943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.887213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.887909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.889939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6ee2a-acc2-4220-a156-c51ba6ff9a68 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:20.903772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:20.904679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:20.905473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:20.905872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:20.906610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:20.908102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.001338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 232/468] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:20 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:21.002597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ed6a1-4c5e-4e1b-8b26-e32cf7e4b325 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.002895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ed6a1-4c5e-4e1b-8b26-e32cf7e4b325 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.003209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ed6a1-4c5e-4e1b-8b26-e32cf7e4b325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.022891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 237/469] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:20 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:21.023539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-271ed6a1-4c5e-4e1b-8b26-e32cf7e4b325 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:21.024713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.024919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-271ed6a1-4c5e-4e1b-8b26-e32cf7e4b325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:21.140010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 233/470] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:21.142165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e504c5-2607-4677-a3a3-4975c7bf4eae None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.142454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e504c5-2607-4677-a3a3-4975c7bf4eae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.142734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e504c5-2607-4677-a3a3-4975c7bf4eae None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.144135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 234/471] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:21.151981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.151981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.151981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.178116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:21.179349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:21.183718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.207969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ad00d-8473-450d-a2e9-ff7d049fded8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:21.220628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.252118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:21.253246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.253714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.254167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.255113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:21.255425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:21.257784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.276595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 238/472] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:21.277590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.284251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 235/473] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:21.285178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.406436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f46da-806c-4499-a5b0-9f22fc52fa56 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:21.444084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:21.444416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fdbc0-7709-4220-be9b-b178923d8c92 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:21.444886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.445210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.449062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.457243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:21.458714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.461710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:21.481423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:21.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:21.484139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:21.484139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:21.484139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:21.484139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.492530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 239/474] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:21.493695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.494978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:21.495762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.496141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.496550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.497511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:21.500240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.615180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c6bf-c9ec-458d-93fa-a7352e488b3a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:21.625467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 236/475] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:21 2022] POST /identity/v3/users => generated 248 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:21.627986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.679253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.713838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:21.717061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.717061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.717061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.717061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:21.717400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.777755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.sql.core [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 12:30:21.777755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 16 12:30:21.777755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [parameters: {'id': '5dbb4b6eda5c415bab008919d83c9908', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 16 12:30:21.777755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 12:30:21.788447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238119-0de2-4e6b-9e79-9b0158b26791 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 16 12:30:21.806954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return ref.to_dict() Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:30:21.807883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application yield resource Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise self.value Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application session.commit() Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 12:30:21.809412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.flush() Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application compat.raise_( Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 12:30:21.810284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:30:21.811414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:30:21.819200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [parameters: {'id': '5dbb4b6eda5c415bab008919d83c9908', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:21.823210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 16 12:30:21.830941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:21.833237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 240/476] 10.222.0.47 () {60 vars in 1131 bytes} [Sat Jul 16 12:30:21 2022] POST /identity/v3/roles => generated 144 bytes in 315 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:21.833237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.837922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:21.838602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:21.838883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:21.839157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:21.839932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:21.840237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:21.842787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:21.856038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 237/477] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:21.862037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:21.924917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.983799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:21.996216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70a35-215e-4472-98ff-5334c740f551 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:22.021477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3173b7-4cd4-4118-b654-f3bcda9a4edc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:22.068467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:22.069332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:22.069610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:22.069882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:22.085181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:22.094783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:22.095661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:22.096690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 241/478] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:30:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:22.101901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:22.102023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:22.102791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:22.103073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:22.103350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:22.104061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:22.105643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc11203-113f-4adb-b5dd-f014f27ee385 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:22.199457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 238/479] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:21 2022] POST /identity/v3/users => generated 249 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:30:22.239337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc7cd6-96c1-43c6-a134-d95e93703536 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:22.266108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:22.307405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:22.308427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:22.308882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:22.313504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:22.314631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:22.315155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:22.316725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2577d7-1fae-45c6-96a1-ce6fee9832bc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:22.328444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 242/480] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:23.377414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761509e-92e0-473b-9b86-e8eb0e6bc71c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.377913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761509e-92e0-473b-9b86-e8eb0e6bc71c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.377987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761509e-92e0-473b-9b86-e8eb0e6bc71c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.379609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 239/481] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:23.384851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.385296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.385485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.413063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:23.413063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:23.436871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f3ee-f8b4-4d6d-b530-78483d9efb62 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.437165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f3ee-f8b4-4d6d-b530-78483d9efb62 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.437550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f3ee-f8b4-4d6d-b530-78483d9efb62 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.442293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 240/482] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:23.443594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.443882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.444162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.468089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:23.472652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:23.584507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 241/483] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:23.600999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 243/484] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:23.613740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:23.645352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:23.708584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:23.775512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe535-9cf4-412d-a946-0cc32b080007 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:23.819749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:23.820897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.821579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.822043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.851109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:23.860017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:23.860881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:23.861984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:23.864417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 242/485] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:23.870849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:23.902069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a5929-d6d6-4077-a9fa-9abec5111db3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:23.932659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:23.974820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:23.975676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:23.976055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:23.976589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:23.989185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:24.002426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:24.003534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:24.004518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:24.009059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 244/486] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:24.012285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.124164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27892e75-d11a-450e-80ad-b07b215c1f0c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.148741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700676c-071b-4b15-8a8c-76f9389e6f2e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.179210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.196066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.197072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.197550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.198015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.199059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:24.199581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.202112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d65-debe-4945-80c9-5b83742ee8a0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.216220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.217495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.217782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.218056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.218436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 243/487] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:24.219146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:24.219458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.225072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184f8f9-c29d-4dba-816f-497ff1b6c03f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.225629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e04b-ad50-4315-8c33-9d2bc6b823b0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.226131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e04b-ad50-4315-8c33-9d2bc6b823b0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.226946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e04b-ad50-4315-8c33-9d2bc6b823b0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.228478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 244/488] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:24.229371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e04b-ad50-4315-8c33-9d2bc6b823b0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.239217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 245/489] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:24.240728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.241285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.241758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.265810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:24.267907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:24.303001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.357160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 246/490] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:24.358218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.367489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e04b-ad50-4315-8c33-9d2bc6b823b0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.411835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.413321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.413770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.414179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.428529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.430407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:24.454065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:24.454969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:24.455813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:24.455813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:24.455813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:24.455813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:24.463309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 245/491] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:24.464348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.500218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee31296-abdc-4c7d-96e1-6bf4e6a0136b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.514751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.548677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.550553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.550976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.551407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.565838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.567303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.587873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059d3a7-eb4a-444e-8b2f-19477a5e72a3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:24.594141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:24.595010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:24.595857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:24.595857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:24.595857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:24.595857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:24.600083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 247/492] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:24.601663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.626686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.629534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.630333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.630613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.630887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.643417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:24.653113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:24.654013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:24.655802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:24.660793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 246/493] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:24.663377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.739255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ade74-3c1f-4ce5-bf3e-a6fc3a2c27ed None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.778988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.780777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.781472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.781752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.782023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.782494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b1772-82d1-4e70-865b-ddd753e5eec8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.782772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:24.783073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.784607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.801417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 248/494] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:24.802288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.820853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.821557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.821931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.822104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.822840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:24.823197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.824491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941c8d-8667-4dff-a075-85abe2709939 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.837567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 247/495] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:24.838824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3789-15cf-4928-b702-02436c446bb9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.839110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3789-15cf-4928-b702-02436c446bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.839384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c3789-15cf-4928-b702-02436c446bb9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.844855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 248/496] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:24.844855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c3789-15cf-4928-b702-02436c446bb9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.892902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:24.915904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484a01e-ce4d-4bd8-9ba8-b9bd1d50f79f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.950365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.950996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.951878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.952154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:24.952272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c3789-15cf-4928-b702-02436c446bb9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:24.952901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:24.953244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:24.954614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:24.962831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 249/497] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:24.963449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:24.992553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:24.993373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:24.993652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:24.993917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.002056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.003486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8f1717e8-4e53-40bc-877a-ded613ec0080 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:25.019491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:25.020609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:25.021553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 249/498] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:25.022111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e7cd-2a53-4f0e-92c0-2e54fdf335ba None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.022111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e7cd-2a53-4f0e-92c0-2e54fdf335ba None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.022482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657e7cd-2a53-4f0e-92c0-2e54fdf335ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.039221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f657e7cd-2a53-4f0e-92c0-2e54fdf335ba None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:25.040319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f657e7cd-2a53-4f0e-92c0-2e54fdf335ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:25.054218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.066345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3d19-258b-4612-b49a-e82280b1e797 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.101883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.102699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.102982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.103261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.108100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 250/499] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:25.109440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1970d-d42a-405a-a33f-61307f2394d4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.109729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1970d-d42a-405a-a33f-61307f2394d4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.110005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1970d-d42a-405a-a33f-61307f2394d4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.110999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.113090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 251/500] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:25.113090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1970d-d42a-405a-a33f-61307f2394d4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.113290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.122162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:25.137333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:25.138510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:25.140264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:25.141076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 250/501] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:25.142042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.225806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1970d-d42a-405a-a33f-61307f2394d4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.247853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3e34-69ff-4936-94be-8cc5e426aaf2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.259353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.260139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.260421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.260700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.270918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.272371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.280698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.281396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.281680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.281956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.282788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:25.283102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.284509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-58df6d83-d22b-4495-bf7e-ad2380cc1544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:25.289688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:25.290503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:25.291369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:25.291369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:25.291369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:25.291369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:25.291369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 252/502] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:25.291980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.292267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.292543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.298749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.311079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:25.320632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:25.323662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 251/503] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:25.324418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.378743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 253/504] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:25.379576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.388547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.439675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425448-f45d-474a-981c-18f09f6308b3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.474335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.475126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.475403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.475679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.478480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb19f0-b484-408a-86ef-720d0d7302cc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.485895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.487327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:25.505064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:25.506312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 252/505] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:25.507151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.510118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.517279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.520991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.520991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.520991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.520991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:25.520991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.521340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.557778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 254/506] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:25.558527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.613062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a610979-538e-4a5b-8fc0-b7e179f3e541 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.624327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.649665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.650327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.650607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.650884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.651603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:25.651917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.653346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.659198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a3070-2f92-469f-bf73-9761bfefcaf4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.664990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 253/507] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:25.670221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.695061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.695705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.695988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.696268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.753680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.764469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.767104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.770784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b0436-5118-43a6-9c79-11e98c997251 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.805354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.806147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.806481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.806759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.817668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:25.825221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:25.826878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:25.827677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 254/508] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:25.828591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.856141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.868246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 255/509] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:25.869116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:25.929171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b34c3-6de9-4533-b54d-09c61435e6e1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.960268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:25.963401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:25.964064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:25.964349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:25.964626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:25.965377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:25.965687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:25.967727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:25.970113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2a00c-c3cb-4655-8df7-7a1dad2e5d49 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:25.979607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 255/510] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:25.980407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.004264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.004911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.005224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.005498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.068761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.073227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.074681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.080942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf5dbf-c60b-477f-85f4-9bc98c3c3451 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.115536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.116322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.116606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.116883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.123918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.125400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:26.139874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:26.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:26.141855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 256/511] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:26.142416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.167921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.183583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 256/512] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:26.183583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.249723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c9ad-4522-439a-bdbc-b06311522b31 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.278546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2556-9ca8-4b6e-b8ac-90eef7342712 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.284484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.285184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.285469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.285751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.286497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:26.286802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.288176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.296422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 257/513] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:26.297145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.311073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.314108 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.314922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.320275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.320275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.325915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.327993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:26.345154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:26.346123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:26.346979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:26.346979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:26.346979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.346979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:26.346979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 257/514] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:26.347616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.390481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.405347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf87e1a-c49d-48b2-b9a1-5cd022f55a01 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.438530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.439196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.439477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.439760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.440462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.441918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.446977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f9764-18af-4c80-9bdf-5544f96120de None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.480107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921128-60f7-4489-a8bd-174309572264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.480763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921128-60f7-4489-a8bd-174309572264 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.481056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921128-60f7-4489-a8bd-174309572264 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.481329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921128-60f7-4489-a8bd-174309572264 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.482149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58921128-60f7-4489-a8bd-174309572264 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:26.482503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58921128-60f7-4489-a8bd-174309572264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.484029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58921128-60f7-4489-a8bd-174309572264 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.500873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 258/515] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:26.501691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921128-60f7-4489-a8bd-174309572264 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.543528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 258/516] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:26 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:26.544414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.611568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.615015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921128-60f7-4489-a8bd-174309572264 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.629326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.653463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.654321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.654602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.654883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.666041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.667522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.668428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15594653-77ef-4e6a-8d72-bf51db7e1737 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:26.687508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:26.688446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 259/517] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:26.696135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.715136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.715941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.716216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.716485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.729722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.731228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:26.753329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:26.754225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:26.755071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:26.755071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:26.755071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:26.755071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:26.755071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 259/518] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:26.755330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.816397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ee3c-eb55-481a-8ba2-11834f94f120 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.856469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.857976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.858474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.859212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.860291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:26.861142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.863079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.863613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ee08-8127-4553-9fa0-bb0fc71f2f91 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:26.877380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 260/519] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:26.878973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:26.882089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.898613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:26.899069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:26.899723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:26.900011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:26.900296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:26.975760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:26.977244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:26.997559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769e75-66ba-46af-bb02-8323317703d1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.042997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.043662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.043954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.044233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.044987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.045327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.046705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.064218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 261/520] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.064218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.088939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 260/521] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:27.088939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.107823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.143401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.181317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c012a6-fbcb-4b4a-bb52-2b3b39588f28 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.206203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37fd18-9b62-4c21-b66a-3a09af382171 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.218117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.219075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.219361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.219635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.220360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.220667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.222080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.236343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 262/522] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.237537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.253943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.254741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.255019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.255290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.262243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.263871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:27.278071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:27.279613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 261/523] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:27.281140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.350068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549906c-36ce-4a45-b1d7-dbe288839a62 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.370054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.382422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.387244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.388025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.388305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.388663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.393462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2375-f2c9-403f-97fb-6577885c4205 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.395262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.396701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:27.411600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:27.412541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:27.413441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 263/524] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:27.413903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.432724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.433566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.433830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.434102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.446975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.448378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:27.466008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:27.467521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 262/525] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:27.469500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.535869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fdf01-289d-4765-a1ae-6d63e83fd67e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.573548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.574337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.574618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.574882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.580756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db244231-78f0-4e6c-8889-e6488446a57c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.583625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.585100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:27.600714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:27.601682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:27.602573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 264/526] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:27.603815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.613843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.619854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.620515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.620835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.621143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.621945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.622252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.624244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.642906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 263/527] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.643924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.667985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.722230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76a17-e245-4121-bcdd-4cc11c34ee42 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.761659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a023d-c7b8-4c17-97ac-7a4fe84188b1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.762264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.762932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.763212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.763854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.765679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.766000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.767442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.794964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 265/528] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.794964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.798677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.799343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.799623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.799895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.800610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.800912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.802326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.814377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 264/529] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.815074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.868679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.911607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12d1a-6355-46d0-8264-373a6bd24d96 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.923592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ca05-fe46-4921-a46b-75a9a3f6d968 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:27.927117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:27.948645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.949338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.949623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.949893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.950728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.951689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.953165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.958850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:27.959499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:27.959786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:27.960060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:27.960891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:27.961235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:27.962649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:27.973456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 265/530] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.973456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:27.977462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 266/531] 10.222.0.47 () {56 vars in 1139 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:27.978590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.091829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027e5b-46c1-49c0-b780-5af2d7044b67 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.101478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e384-cd7f-4a0c-a893-b59e2b343bcf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.118059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:28.143737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.144534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.144816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.145107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.153741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.155246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.156492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.157234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.157515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.157785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.158625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=eff6db7c70f64a649a64313389f7e9bf) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:28.158937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.162360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.170670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 266/532] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/role_assignments?user.id=eff6db7c70f64a649a64313389f7e9bf&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:28.171847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:28.185435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:28.187030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:28.187939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:28.190636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 267/533] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:28.196589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.286926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:28.305817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb65db-7717-4d04-bdf4-b333459a970d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.349584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.351113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.351614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.352085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.411234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:28.441446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.442525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0091ff-c6c1-4f4a-b9c2-7b1683b6f9b1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.443453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.479321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.479982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.480268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.480545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.481369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:28.481685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.483868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.496895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 268/534] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:28.503446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.573396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 267/535] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:28.584716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.674550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:28.687572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a34e5f-6675-4f3c-b329-e6f9818fea07 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.732130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.732844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:28.734096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.734382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.734637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.735472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d10255724102414ca184fac122fd1e09) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:28.735784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.737263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b86b6-cb07-47a4-9266-99a2424a13b1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.749081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 269/536] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/role_assignments?user.id=d10255724102414ca184fac122fd1e09&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:28.749081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8cb29-aef0-41e7-9598-5d7c54c94d15 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.749081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8cb29-aef0-41e7-9598-5d7c54c94d15 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.749081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8cb29-aef0-41e7-9598-5d7c54c94d15 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.749081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 270/537] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:28.760500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.760819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.761085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.791567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65845bb7-2fe5-467b-8db0-6da03a8b716b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:28.794330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:28.796132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:28.857176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:28.858665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:28.858952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:28.859228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:28.885066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:28.885066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:28.896533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 271/538] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:28.913065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:28.929619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:28.930518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:28.931359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:28.931359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:28.931359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:28.931359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:28.931359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 268/539] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:28.946111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.013143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:29.030212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073e8a4-41a4-4a93-8312-9e6b4d4407e2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.068352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:29.069177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:29.069465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:29.069742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:29.080907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:29.088866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:29.091013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:29.093209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:29.094921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 272/540] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:29.105065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.227442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189ff22-7d7f-4451-bf8e-c390fdb4aff3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.287393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:29.296997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ce2c-adae-403f-92e7-8cd5e219d826 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.300585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:29.301592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:29.302050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:29.302404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:29.303140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:29.309657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:29.311350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:29.333072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 269/541] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:29.350103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.354492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:29.355173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:29.355455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:29.355726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:29.356469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:29.356772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:29.358772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:29.371587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 273/542] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:29.374704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.392861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:29.581163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a7189-9e61-4245-a6d1-238c431cb00a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.607316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d559d5c-f932-4705-a643-01c7da6f45e4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.633597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:29.635331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:29.635621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:29.635899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:29.649063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:29.649352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:29.677531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:29.678828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:29.679699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:29.679699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:29.679699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:29.679699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:29.679699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 274/543] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:29.694586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.756257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:29.757440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:29.758632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:29.759138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:29.778398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:29.782114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:29.816755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:29.817706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:29.818544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:29.823122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 270/544] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:29.838329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:29.840744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:29.931106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47949-fddd-4699-8daf-f77fe9d95a0a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:29.965065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:30.029685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.030839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.031267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.031700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.032570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:30.033048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:30.034578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:30.064040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 275/545] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:30.069089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:30.069985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f538b-d62d-4708-bdc4-e7d83f71cb27 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.143415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.143635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.144137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.144137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.144903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:30.153501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:30.155673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:30.192872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 271/546] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:30.203120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:30.243035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c530c9-bd22-46a5-a43f-6c05a41bb412 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.279590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:30.296801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.298056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.298594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.299317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.310066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:30.312436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1fe6c3e3-eb8b-4c3b-8d7b-28550491bbf2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:30.329494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:30.331031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:30.332033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:30.334678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 276/547] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:30.336360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e578b-fa3c-48a3-a878-09981006770e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.336857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e578b-fa3c-48a3-a878-09981006770e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.337378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e578b-fa3c-48a3-a878-09981006770e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.338856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 277/548] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:30.346599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e578b-fa3c-48a3-a878-09981006770e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:30.429741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782d00-be2f-4c9b-b469-7f1fbe3a6a4c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.450664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:30.476836 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e578b-fa3c-48a3-a878-09981006770e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.497412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.498555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.499455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.499938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.502369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=246c8e2b8aec415496a87067875975a6) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:30.504231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:30.508313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883598c-63b7-4fb8-b859-be6912ab3c2f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:30.529942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 272/549] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/role_assignments?user.id=246c8e2b8aec415496a87067875975a6&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:30.530708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.531008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.531287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.541160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.541160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.541160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.541160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.541666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:30.541666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:30.542920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe3e40-0855-4007-85c9-d47d360d1992 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:30.568364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:30.569688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:30.576970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 278/550] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:30.580672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa22694-5270-4944-8515-c3b317f14b89 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.580672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa22694-5270-4944-8515-c3b317f14b89 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.580672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa22694-5270-4944-8515-c3b317f14b89 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.585196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 279/551] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:30.586640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.586927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.587204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.617185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:30.618407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:30.692199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:30.722714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 273/552] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:30.724317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:30.757129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 280/553] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:30.762567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:30.869422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9d774-a18c-4610-ae96-78d3f6922528 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.871889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:30.884665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33333a-7c64-4a9b-9b15-807afefd38e1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:30.917197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.919143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.919465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.921176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.927834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:30.928640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:30.928927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:30.933059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:30.942543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:30.952561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:30.953530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:30.954451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:30.961065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 281/554] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:30.961065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.001063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.052695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.054339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.101113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338416b1-1dd9-485c-85a2-ce198065843a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.142599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.143540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.143827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.144105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.155840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:31.164292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:31.165227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 282/555] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:31.166636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.249517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 274/556] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:31.250621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.254893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.271833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.289838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39fee-5db8-4a69-8ba9-637086abafcb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.333780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.335335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.335620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.335894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.336653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:31.336965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.338516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.351204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 283/557] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:31.352249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.387800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dbb7-89a1-4de7-aedd-56b8a8db9bc7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.433906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.436920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.437224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.437495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.438242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:31.439976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.442035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dcc59-a9c6-4baf-abae-676b57aab3d9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.458591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 275/558] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:31.458591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8028-8508-4986-995a-4022b06e6dce None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.458591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8028-8508-4986-995a-4022b06e6dce None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.458591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8028-8508-4986-995a-4022b06e6dce None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.458591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 276/559] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:31.459455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8028-8508-4986-995a-4022b06e6dce None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.476159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c48b8-c69a-44df-abcb-5c3f9b0d0347 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.523054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.524118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.524599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.525107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.544413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.545768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.548373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.558658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:31.568907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:31.569964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:31.570844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:31.570844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:31.570844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.570844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:31.577666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 284/560] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:31.578781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.585872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8028-8508-4986-995a-4022b06e6dce None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.629761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.630704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.630999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.631269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.645831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.647371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-382b862f-dc1b-4dfe-9e02-23445f0b69f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:31.667904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:31.668963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:31.670703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:31.670703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:31.670703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.670703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:31.676353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 277/561] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:31.677744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.678033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.678310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.696711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:31.697931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:31.719704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015bd9-c3fa-4dd6-8135-44ce872f06b0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.773146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.774282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.775141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.775604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.791705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.793221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.800985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 278/562] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:31.802413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:31.818179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:31.819111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:31.820088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:31.820088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:31.820088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:31.820088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:31.820088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 285/563] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:31.820787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:31.867002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.891375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:31.934339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8c425-3cae-492e-adba-7a28acbe067a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.956861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c1dd-cce4-417b-b122-0222ae93b3da None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:31.984567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:31.985619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:31.985896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:31.986217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:31.986943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:31.987398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:31.988991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:31.999986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.000943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.001451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.001924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.002814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.003266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.003494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 279/564] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.004420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.005259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.022783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 286/565] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.024022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.131166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57851d-3068-40b8-bfc2-accb7ba1a4cd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.139826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.154802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd555e-2ba3-420d-8651-0d88685daf8c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.171969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.173623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.174302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.174578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.174846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.181194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.181404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.182787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.199329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 280/566] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.200380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.201243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.201533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.202122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.204704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.217558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:32.226522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:32.227485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:32.228354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 287/567] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:32.229163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.349079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe27af-71f6-421c-9ed1-207bb060f074 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.370090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a84a-d8c7-4d85-9395-4d27c87363de None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.397167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.398339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.398827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.399297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.410099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.411697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.421466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.422383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.422787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.423166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:32.428760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:32.429838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:32.430769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 281/568] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:32.431256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.440234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.442300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:32.458786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:32.459878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:32.460812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 288/569] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:32.465424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.471243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.502936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.564501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e30107-c1e2-4adb-974f-bcb5c4ec3637 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.584702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf75d8-fe65-474b-ac24-f889ddb4b866 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.607965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.609280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.609783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.610671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.621864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.624196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.630578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.631284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.631740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.631847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.632906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.633264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.634765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:32.642689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:32.643623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:32.644630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:32.645200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 289/570] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.645739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.647183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 282/571] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:32.648153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.751276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.759110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:32.763859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef0d63-cba7-4355-959c-53de2e6e4430 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.766695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444108-25dd-4038-89b9-b46a325672d3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.799752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.809423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.809423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.809423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.809423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.810709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.810804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.810878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.810878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.810878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.811070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.811149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.811149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.811149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbbbda-1a2f-45fc-963b-47471d5cefba None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.820227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 290/572] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.821958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8362-5bf6-4bb9-9a4a-0e6efe66d464 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.822448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8362-5bf6-4bb9-9a4a-0e6efe66d464 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.822727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8362-5bf6-4bb9-9a4a-0e6efe66d464 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.824131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 291/573] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:32.824699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea8362-5bf6-4bb9-9a4a-0e6efe66d464 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.829348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 283/574] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:32.830352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:32.946545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea8362-5bf6-4bb9-9a4a-0e6efe66d464 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.946718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356481-3004-47ec-829a-8c9845b01f9b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:32.983821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.984618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.984896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.985205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.985644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:32.986314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:32.986589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:32.986857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:32.987685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:32.987990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.989433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:32.996159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:32.998368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.013206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 284/575] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:33.013206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9e973e3b-8ad2-49f9-be1a-1e3bfa6f09e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:33.017000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:33.017993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:33.018902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:33.018902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:33.018902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.018902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:33.020519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.021335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 292/576] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:33.022454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.022958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.023280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.035432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.041270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:33.042763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:33.122659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 293/577] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:33.123604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.135464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea15f17-a389-4427-88c6-8d806c5a461e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.176286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.177491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.177924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.178370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.179437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:33.179960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.181855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.189382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 285/578] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/role_assignments?user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:33.190449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.273384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34b8f6-a9df-43f7-806b-797592b8425e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.293046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.329645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.330862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.331496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.331770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.337612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.338669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8355369-b73a-4a25-8d78-beff8e88abd5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.349759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.352537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:33.372328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:33.374175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:33.375251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:33.375251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:33.375251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.375251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:33.381849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 294/579] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:33.383380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.387959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.389351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.389638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.389913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.404227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.406772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:33.431098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:33.432052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:33.432956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:33.432956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:33.432956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:33.432956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:33.437411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 286/580] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:33.438717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.522887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015b800-4c57-4e18-89d0-7d65ea483443 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.561447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.562205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.562484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.562759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.566454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:33.566653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.568074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.576886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1872f-dc57-458f-84aa-037090cc7e60 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.584976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 295/581] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:33.584976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.612506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.628024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.628823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.629150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.629420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.641795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:33.650626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:33.651584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:33.652423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 287/582] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:33.654400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.670853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.741199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cffa-d823-4308-a1fd-695e4c8670bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.778190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc307de-328f-4976-9d6c-2777cbcf3733 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.779121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.779814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.780397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.780689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.781565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:33.781891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.783488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.802803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 296/583] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:33.803724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.817050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.817835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.818096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.818367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.819095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:33.819400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.821150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:33.834199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 288/584] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:33.839695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:33.889629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.936474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497933d-e154-468e-843f-cd7b131408ce None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.947123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:33.960881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d27f4-31f7-4de4-b6e5-93da9c36c14f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:33.979788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:33.980591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:33.980869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:33.981175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:33.991738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:33.993906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.004776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.005705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.006067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.006418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.007418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:34.007791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.009334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:34.010330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:34.011235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 297/585] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:34.012945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.023907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 289/586] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:34.026906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.147321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8630be9-f573-4b02-abb2-a63ff03eb409 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.160578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3f3f-90e5-4ea2-9dbb-94f99d3a3ece None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.162913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.187588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.188396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.188673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.188941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.196211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.197815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.211444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.212244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.212525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.212799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:34.216960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:34.217887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:34.219029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:34.220377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.222139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.223542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 298/587] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:34.227213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:34.240298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:34.241254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:34.242179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:34.244802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 290/588] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:34.244802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.263231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.352948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbb1-c14e-4244-a0a2-14d9343afa6b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.368478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca290-a1d3-4a30-bd7f-eb23ebfa3e9e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.399668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.400721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.401013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.401351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.402166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:34.402625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.404241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.408391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.409224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.409733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.410003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.421676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.423094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.432526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 299/589] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:34.433514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:34.443443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:34.444354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:34.445849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:34.445849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:34.445849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:34.445849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:34.445849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 291/590] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:34.446178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.469695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.540568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.558544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eafb5-035b-4058-b484-dee9da9f94d1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.578974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbea0f-bd7c-4d65-b168-06330de3e4cc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.609855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.609855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.610581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.610581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.611250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:34.611555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.612976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.621767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.622864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.623364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.623947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.624904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:34.625245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.626292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 300/591] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:34.627007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.630016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.652616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 292/592] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:34.654638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.762072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.764318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5cb1-670f-42c2-a445-67698c740e30 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.787159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f241de-6b85-4679-8886-e962563713d8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.804193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.805106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.805485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.805793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.821155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.822925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.829599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:34.830636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:34.831082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:34.831293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:34.831574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:34.832312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:34.832621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:34.834413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:34.845470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:34.863985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 301/593] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:34.864901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.865289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 293/594] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:34.865289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:34.978827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dbf1c-0721-407f-9197-fd4a0713aad6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:34.982715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4295b-84dd-4fcf-a350-bf9fa6cd9c63 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:35.020632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.021676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.022029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.022381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.022976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.024015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.024461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.024871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/eff6db7c70f64a649a64313389f7e9bf/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.029346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:35.040026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=0faa6a54416f420bbdec92832d404efd, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.043042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.048709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=eff6db7c70f64a649a64313389f7e9bf, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.050359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.132519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 302/595] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:34 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:35.133386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:35.168201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 294/596] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:34 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/eff6db7c70f64a649a64313389f7e9bf/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:35.168201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:35.270321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:35.370235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4533-44a6-47f3-b3dc-6a8901f7da82 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:35.374880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bfd9c-49ea-4864-8673-9bb4b43b6689 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:35.423605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.424412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.424690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.424961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.426697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.429549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.430042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.431751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.432840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:35.433367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.436139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.436801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b28555-9a45-49d2-8297-67657ef49c75 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.437977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.453403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 303/597] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:35.455930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c429ef5-8e62-4e00-abbf-2bd6fab3a6a8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.456408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c429ef5-8e62-4e00-abbf-2bd6fab3a6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.456818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c429ef5-8e62-4e00-abbf-2bd6fab3a6a8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.465349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 304/598] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:35.466534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c429ef5-8e62-4e00-abbf-2bd6fab3a6a8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b6da6fdd-fa46-4fec-b177-7b02e00f17ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:35.466665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:35.467551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:35.468769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 295/599] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:35.469472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.469909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.470535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.499220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:35.502462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:35.585705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:35.629711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 296/600] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:35.631039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:35.661726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c429ef5-8e62-4e00-abbf-2bd6fab3a6a8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:35.682855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:35.738635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.739570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.739972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.740251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.752170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.755055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:35.783877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:35.784925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:35.785889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:35.795936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 305/601] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:35.798260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:35.819933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e0763-7e56-45fc-a579-ba9ca968acff None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:35.921064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:35.921064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:35.921064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:35.921064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:35.921887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:35.921887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:35.921887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:35.958295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 297/602] 10.222.0.47 () {56 vars in 1139 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:35.959445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.019160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967e6d-10c4-4219-bb28-ccd829f667c6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.027868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:36.097294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.098444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.098943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.099520 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.125415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:36.126243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1f4ae-8fc2-4c95-9364-1168ec6938c2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:36.134406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:36.135359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:36.136203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:36.139553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 306/603] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:36.140768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.187024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:36.193970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.195646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.200233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.200233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.200233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:36.200233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.202270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.224735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 298/604] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:36.226046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.303841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b29541-f355-48e7-9af1-bec61b660366 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.361244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:36.371316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.372130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.372923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f901b2c-0837-4564-8900-ab6513649c68 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.373068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.373388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.385975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.387326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:36.409618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:36.410589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:36.411527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:36.411527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:36.411527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:36.411527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:36.413910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 307/605] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:36.414112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.428132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.429223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.429719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.430202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.431327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:36.431861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.433607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.450220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 299/606] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:36.451611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.517410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:36.538901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49a9d7-7150-4618-89d2-626b67e824df None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.560860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55217f-cd1b-4162-ad15-368ae6938924 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.585282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.586177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.586458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.586734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d10255724102414ca184fac122fd1e09/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.601908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.603007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.603545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.604045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.604690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=d10255724102414ca184fac122fd1e09, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.605239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:36.605749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.606174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.616432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.620822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 300/607] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:36.621841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.644201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:36.707954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 308/608] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:36 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d10255724102414ca184fac122fd1e09/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:36.709002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.783474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76bfe-b915-4fed-a1d1-56f10e6b0252 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:36.854844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:36.855654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:36.859926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:36.862932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:36.888653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:36.890246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:36.925906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:36.927369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:36.928264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:36.928264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:36.928264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:36.928264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:36.929290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 301/609] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:36.941111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:36.966656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b689b-5343-4d9a-a501-906b3a17edfc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.020672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:37.033999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.038403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.039717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.040005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.049911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:37.055277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.057665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:37.077450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:37.078395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:37.079703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:37.081919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 309/610] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:37.089426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:37.105923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f9f1-7434-4c2f-bb70-7986055fcce7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.146681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.147342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.147616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.147880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.148594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:37.149548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.150942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.183939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 302/611] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:37.194103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:37.336858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d457c94-8435-4f9c-9b56-2af890cc058a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.351988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b563447-5569-48ff-bc2b-1fe959412338 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.397704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:37.407474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.408395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.408737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.409505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.417787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.420083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:37.437643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:37.438676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:37.439672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 303/612] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:37.444676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:37.478692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.479368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.479651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.479925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.480725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:37.481059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.483023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a9f3f-93f3-485b-a0ec-b0ffefec7daa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.504791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 310/613] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:37.506971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1063dd-7add-4d01-9242-496bff99c8fb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.506971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1063dd-7add-4d01-9242-496bff99c8fb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.506971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1063dd-7add-4d01-9242-496bff99c8fb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.512125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 311/614] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:37.514881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1063dd-7add-4d01-9242-496bff99c8fb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:37.591714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1c3e-2235-40cc-8486-870d218b74ba None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.601340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:37.639567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.640570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.641032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.641511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.642566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:37.643019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.644557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696bf2b-ad6c-4c6c-ae61-765cd17857d4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.665399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1063dd-7add-4d01-9242-496bff99c8fb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.670803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 304/615] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:30:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:37.670803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162956d-b642-4a75-9a7b-bb7782f717ba None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.670803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162956d-b642-4a75-9a7b-bb7782f717ba None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.670803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162956d-b642-4a75-9a7b-bb7782f717ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.689936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1162956d-b642-4a75-9a7b-bb7782f717ba None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:37.691462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1162956d-b642-4a75-9a7b-bb7782f717ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:37.720333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.721534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.722027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.722369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/246c8e2b8aec415496a87067875975a6/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.737073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:37.740214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=246c8e2b8aec415496a87067875975a6, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.741735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036bf52-074c-45b8-8467-6befc80f4772 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:37.766195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 305/616] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:37.767528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06369-7ab1-4ebc-bc80-83ad9648500f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.767816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06369-7ab1-4ebc-bc80-83ad9648500f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.768088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06369-7ab1-4ebc-bc80-83ad9648500f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.769902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 306/617] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:37.770606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06369-7ab1-4ebc-bc80-83ad9648500f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:37.845863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 312/618] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:37 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/246c8e2b8aec415496a87067875975a6/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:37.848504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.848819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.849128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.883760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:37.886254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:37.923353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06369-7ab1-4ebc-bc80-83ad9648500f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:37.963737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:37.964554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:37.964825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:37.965175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:37.980481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:37.982152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:38.004523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:38.005597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 307/619] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:38.007295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.043103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.067541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 313/620] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:38.068408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.135327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.161873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535bde4-2b87-4def-8399-d5117ee49b1d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.226446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.227583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.228053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.228509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.252113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:38.263380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:38.264380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:38.265311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:38.269985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 308/621] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:38.269985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.276848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad4f8-23d6-4db9-b8d5-57b02d4134e5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.354447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.355119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.355394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.355662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.356384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:38.361060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:38.361060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:38.388751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 314/622] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:38.389560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.457870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.470374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec661c3-3861-4794-b2fe-736c4c1bc264 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.521445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.529800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e5b6-dd0f-4a29-af79-290a839d055a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.547070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.547882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.548191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.548468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.576550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:38.586801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:38.589976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 309/623] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:38.591373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.598125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.600296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.600788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.601287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.602319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:38.602871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:38.605335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:38.619270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 315/624] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:38.620414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.745998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489d5f6-9c5f-4c5c-95d5-b016c5379202 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.752828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e992da-154d-43a9-a2da-3ba5275be2f7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.791041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.791838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.792120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.792394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.802242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:38.803627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:38.811995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:38.812995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:38.813492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:38.814012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:38.814950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:38.815450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:38.817107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:38.827356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 316/625] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:38.828454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:38.836542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:38.837530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:38.838417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 310/626] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:38.838928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:38.848516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.851570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:38.946886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f322-3d60-40aa-850c-0ca6b39de98a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.998289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae391a60-900a-43d7-b13c-ded76dc5f087 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:38.999973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.000949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.002892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.003334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.024424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.027110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:39.052406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:39.053411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:39.054330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:39.054330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:39.054330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:39.054330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:39.057177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 317/627] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:39.058197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.080334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.081291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.081577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.082377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.107891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.108086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:39.133959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:39.135012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:39.135899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:39.135899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:39.135899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:39.135899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:39.138307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 311/628] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:39.139323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.150127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:39.192248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6714825-ae80-42ac-9a3f-f59cf138d4f9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.229609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:39.237166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.237931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.238285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.238630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.239754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:39.240016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.241722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ce1ac-0dc5-4365-a051-2bba581e41af None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.256362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 318/629] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:39.257245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605117be-6309-43c5-bb87-4ce490b71e54 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.257532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605117be-6309-43c5-bb87-4ce490b71e54 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.257803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605117be-6309-43c5-bb87-4ce490b71e54 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.259369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 319/630] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:39.260064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605117be-6309-43c5-bb87-4ce490b71e54 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.289691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580f7c-0efa-49d2-bdd4-7213d28fbaf4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.339483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.340160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.340416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.340686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.341973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:39.342265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.343875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.359051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 312/631] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:39.360521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.397693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605117be-6309-43c5-bb87-4ce490b71e54 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.429756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:39.440613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.441368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.441704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.441981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.442750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:39.443239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.444715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013b87f-a925-4f5c-90ad-96e53c81e79b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.458918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 320/632] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:39.460224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.460515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.460792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.480179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:39.481939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:39.492882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ecb3-cb81-490a-b3c8-21550d416231 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.515953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:39.531815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.533078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.533215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.533569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/6d7e37eb8b3e44d5a7f2be9af4a02c1f/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.551437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=6d7e37eb8b3e44d5a7f2be9af4a02c1f, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.553016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.565672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 321/633] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:39.566540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.641568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 313/634] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:39 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/6d7e37eb8b3e44d5a7f2be9af4a02c1f/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:39.642776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.721683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494144e-7c0e-45c7-90e0-745f71b734df None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.749444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:39.775784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.777935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.777935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.777935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.788454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.790128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:39.807609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:39.808623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:39.809654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:39.817063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 322/635] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:39.817063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:39.865767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d147c1-b4c0-4e90-9148-5dd7222871b0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.941063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:39.941063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:39.941368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335405f-22c2-4b22-a8dd-f15fefe5859e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:39.941454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:39.941454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:39.966711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:39.972946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:39.993142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:39.994196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 12:30:39.995242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:40.001452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 314/636] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:40.007043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.017060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.017060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.017060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.017060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.037658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:40.052128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:40.053420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 323/637] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:40.054481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.063443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.102400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.174864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3817-dc4b-4d3d-b839-2fd215290597 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:40.182866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040dbc-0be2-4cd6-8e9a-8c6089679797 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:40.221576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.223434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.223434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.223434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.225112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:40.225555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:40.226385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.226809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6057cb-d300-41f6-8585-3264d1275633 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:40.227531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.228048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.228788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.229875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:40.230405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:40.232068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:40.260162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 315/639] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:40.260378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 324/639] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:40.263120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b785e8-75b2-4420-8c5b-8afeb807968d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.263120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b785e8-75b2-4420-8c5b-8afeb807968d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.263120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b785e8-75b2-4420-8c5b-8afeb807968d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.264219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.264815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 325/640] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:40.265540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b785e8-75b2-4420-8c5b-8afeb807968d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.390877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.404975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.413218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509b89-e7e9-4317-9073-da9546f76b91 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:40.416294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b785e8-75b2-4420-8c5b-8afeb807968d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:40.463954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.464637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.464919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.465228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.466573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:40.466885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:40.468644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:40.473383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.474350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.474856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.475382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.483513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 316/641] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:40.483513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.569245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:40.570869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef603baf-7b03-4653-9df2-49d828eb4ff2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:40.617312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870292-0677-40e1-8f5c-9e7d29eea1f5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:40.669919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:40.670615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.670985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.671328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.689717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.696962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.705609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 326/642] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:40.707628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:40.708308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:40.709185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:40.731129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:40.732958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:40.759979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:40.762208 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:40.817344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 327/643] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:40.818872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.912321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 317/644] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:40.921130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:40.979088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:40.986774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a29d93-bd58-457d-89ae-94e67ff71056 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.020984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.036837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187e3ce-55fd-41a6-8115-b73f2f46ca2f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.045003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.047152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.048558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.048991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.061992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.065457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.081430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.082220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.082507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.082776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:41.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:41.102158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:41.103099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:41.103099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:41.103099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.103099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:41.103099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 328/645] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:41.103420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.108038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.110114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:41.131674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:41.132791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:41.134081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:41.134081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:41.134081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.134081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:41.140962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 318/646] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:41.142050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.226619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c2a6a-38a2-4d39-9e5c-d81b1b849f87 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.282534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.286744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb8f3f-c6c5-4bea-aeef-07dccfbd9432 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.290641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.290641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.290641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.290641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.305762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:41.314044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:41.314994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:41.315852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 329/647] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:41.316564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.351881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.352770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.353076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.353384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.368429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.369465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.370914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:41.395507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:41.396631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 319/648] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:41.397744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.449120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5f3ff-5746-4820-93f7-e58b5324de7d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.485020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.485708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.486058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.486245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.486959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:41.487253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.488603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.504694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 330/649] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:41.504694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.624944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5d05-b61b-41ba-be65-c6880640445f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.639571 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.660527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.661212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf639a01-e50e-4ecb-a097-2671ebb6d57e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.662570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.662570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.662570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.662820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:41.663181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.664622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.681503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 320/650] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:41.682971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.697885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.721876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.726934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.727245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.727532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.728305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:41.728609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.730015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.739130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 331/651] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:41.739951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.830769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f528-3222-4aed-8e97-7195b291cf2b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.851524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a31be3-de58-420f-8f79-910335c251e9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:41.865932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.866673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.866950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.867218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.867935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:41.868236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.869624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.886873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 321/652] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:41.888226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.890408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:41.891229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:41.891509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:41.891778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:41.899458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:41.900882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:41.915244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:41.916364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 332/653] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:41.917493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:41.940621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.960597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:41.992302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e5382-a0d6-4eb6-8015-c5769bf0103f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.025194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be573e4-7122-4cb4-9681-43dcb4bcc5c8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.032030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.032900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.033285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.035179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.042010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.043458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-73906b6e-405f-4a70-83ef-3ecaf7b9a0a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:42.057816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:42.058909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:42.060129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 322/654] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:42.060663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ec43-991d-4760-8d6f-02c59ea9a455 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.060663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ec43-991d-4760-8d6f-02c59ea9a455 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.060793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ec43-991d-4760-8d6f-02c59ea9a455 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.062226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 323/655] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:42.062727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8ec43-991d-4760-8d6f-02c59ea9a455 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.072991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.073813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.074101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.074373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.084933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.086414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:42.104607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:42.105745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 333/656] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:42.107263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.166390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8ec43-991d-4760-8d6f-02c59ea9a455 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.173736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.196977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.201488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.202275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.202553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.202822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.206455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9f7a2-298d-4185-909a-8b01ca79745e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.209402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.210823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:42.227616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:42.228706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 324/657] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:42.229666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.243536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.244141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.244419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.244689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.245555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:42.246179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.247964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958bbb-96f2-47b1-913a-d94fb6bcc176 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.266063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 334/658] 10.222.0.47 () {56 vars in 1137 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:42.267089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.267373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.267643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.283760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:42.284932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:42.329947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5437b-5014-4c9b-bd5f-81837e0388c8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.350459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 335/659] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:42.351261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.367535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.368270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.368554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.368827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.369646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:42.369950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.371362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.383539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 325/660] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:42.384329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.411972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.439681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.453631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16bbbe-ad61-41e3-85de-17b566fcce24 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.486431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d39fa-7d53-4fba-a90e-55d8ed6d5da5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.487817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.488472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.488753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.489025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.489760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:42.490066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.491422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.503205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 336/661] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:42.503849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.525714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.526372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.526654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.526929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.527710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:42.528022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.529471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.541148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 326/662] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:42.541931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.605027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc457-82aa-4216-8236-d3f923df3045 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.622927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.638828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.639107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1688e-4da1-414f-a682-2adcf57e3df0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.639642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.639983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.640449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.651876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.673440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.674217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.674503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.674777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.685381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:42.693747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:42.694742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:42.695666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 327/663] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:42.696364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.709389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.710850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.795600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a811-b1a4-4726-81e0-cc4f567707e6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.809691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 337/664] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:42.810522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.828587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.831164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.831820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.832098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.832375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.833239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:42.833555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.835041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.843958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 328/665] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:42.843958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:42.884181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:42.920500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba5a0-fd09-4a80-a709-12e2a209fa95 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.958909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3989c8-ca9c-418f-aa20-17e5cc83f67b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:42.959164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:42.960116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:42.960537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:42.961016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:42.969912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:42.971690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:42.988387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:42.989564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 12:30:42.990646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:42.993318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 338/666] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:42.994237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.002711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.003376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.003658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.003936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.005110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=eff6db7c70f64a649a64313389f7e9bf) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:43.005221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.006595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.015904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 329/667] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/role_assignments?user.id=eff6db7c70f64a649a64313389f7e9bf&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:43.016805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.103033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.144861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af4792-0ac8-4858-909b-846d47d8bf10 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.173021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.176910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e754d47-709e-426a-b09b-fd93204ba8fe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.209721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.209721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.209986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.210084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.211339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:43.211339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.213679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.226608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 339/668] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:43.227471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.236165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.237354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.237850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.238395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.257704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.259868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:43.284678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:43.285716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:43.286675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:43.286675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:43.286675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:43.286675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:43.289422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 330/669] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:43.290725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.393065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4624b-2d48-48ec-8eec-efe86b493aaa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.469088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.469088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.469392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.469655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.470490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:43.473058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.473058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.487243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8550f-c267-4458-a9e5-b25a96c5ceff None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.491952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.504028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 340/670] 10.222.0.47 () {56 vars in 1139 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:43.505004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.531433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.532236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.532520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.532794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.549762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.551218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:43.575975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:43.577213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 331/671] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:43.579038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.583309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.624218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57bd75-dce2-4e05-a952-1a91975a8306 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.664040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.665088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.665642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.666124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.667137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:43.667652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.669855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.685603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 341/672] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:43.686904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.709252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16274c25-9b92-4918-a1cf-e6f3814fafc4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.752515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.753232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.753521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.753803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.755846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.805806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d2a5-2641-4e46-8a1b-95d5b6a63f7a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.835932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.837726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.841502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:43.841621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:43.842436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:43.842797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:43.843060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:43.843980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:43.844102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:43.845430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:43.858845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 342/673] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:43.859634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.953481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 332/674] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:43.954696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:43.964338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cfd9-bafd-4ad2-9997-a0c5b3f40ffd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:43.983992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.001247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.002041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.002327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.002593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.009550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.010990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:44.026158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:44.027082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:44.028593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 343/675] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:44.029131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.069239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcaa43-ed21-4631-9e61-1e3d139cf727 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.072029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.108713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.109922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.110415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.110895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.118553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.120223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.134841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8684-6603-4c39-b347-fcfc208bab0e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:44.136000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:44.136942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:44.138721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:44.141726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 333/676] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:44.142961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.173778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.174564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.174899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.175182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.186689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.188198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:44.207288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:44.208231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:44.209160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:44.209160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:44.209160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:44.209160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:44.209160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 344/677] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:44.209853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.241762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.266626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d45e2b-d7f1-4ad8-8d3f-3ff0ec6c6b28 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.305781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.306929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.307368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.307794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.308730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:44.309979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.311857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.316324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12682b61-8025-487c-ad16-c347bb0e0aa2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.325945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 334/678] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:44.327278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.354695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.355370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.355661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.355936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.356820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:44.357276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.358284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.359554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.378301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 345/679] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:44.379378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.441975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960dbf6-2b2b-4324-9340-9259cef41805 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.474893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.482329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.485206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.485206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.485206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.485206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:44.485206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.487176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.488057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9207-7aa2-4190-b66e-bcef99e8fc4e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.504967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 335/680] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:44.506718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.525877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.526596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.527039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.527290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.528722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d10255724102414ca184fac122fd1e09) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:44.529152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.530640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.537537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 346/681] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/role_assignments?user.id=d10255724102414ca184fac122fd1e09&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:44.538467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.604272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.635642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01fe4c-2957-4b50-be17-86adb316a33e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.668703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc8040-828f-4b5d-82e8-9d110c6f67d6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.678985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.684897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.685524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.685960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.757759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.758824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.759359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.759786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:44.778756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.780218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:44.800695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:44.802119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:44.803279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:44.806247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 347/682] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:44.812485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.821630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:44.823219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:44.835171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.935141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:44.946698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf68d-c370-4b2b-8e7b-dc751a137fc4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:44.954866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 336/683] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:44.960218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:44.996378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:44.997952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:44.998386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:44.999075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.000932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:45.001440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.003024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.019639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 348/684] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:45.029884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.077142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db089dc-e49c-4d5c-abe6-7781b86194e9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.122683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.123902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.124392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.124868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.126384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.138575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.144482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7ead829a-a59c-4b31-a92a-8a5e0ff2b8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:45.163833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:45.164787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:45.165732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:45.165732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:45.165732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:45.165732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:45.165982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 337/685] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:45.167604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5521-9209-42a6-b806-f4f564b0c427 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.168115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5521-9209-42a6-b806-f4f564b0c427 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.168514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5521-9209-42a6-b806-f4f564b0c427 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.170272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 338/686] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:45.172369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7475-5537-42c8-b550-ff7d482ecfe3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.174863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba5521-9209-42a6-b806-f4f564b0c427 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.214228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.217948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.218625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.224899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.224899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.224899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=246c8e2b8aec415496a87067875975a6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:45.224899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.224899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e626f-1ff4-4f79-8797-4a7222f2f590 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.230586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 349/687] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/role_assignments?user.id=246c8e2b8aec415496a87067875975a6&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:45.233659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c0c1-8de9-4e36-978b-8c2a1fddd02c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.234159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c0c1-8de9-4e36-978b-8c2a1fddd02c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.234615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c0c1-8de9-4e36-978b-8c2a1fddd02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.271877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0895c0c1-8de9-4e36-978b-8c2a1fddd02c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:45.274645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0895c0c1-8de9-4e36-978b-8c2a1fddd02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:45.323620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba5521-9209-42a6-b806-f4f564b0c427 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.363665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.364840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.365485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.365763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.366502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:45.366826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.368333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.375552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 350/688] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:45.377279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc04ce4-acfc-4b42-a4d2-859072b7c6ee None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.377756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc04ce4-acfc-4b42-a4d2-859072b7c6ee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.378211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc04ce4-acfc-4b42-a4d2-859072b7c6ee None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.384766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 351/689] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:45.387096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.388147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.388642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.392906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 339/690] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:45.398700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.412573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:45.413609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:45.497653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.508694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 352/691] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:45.511637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.529499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.534700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9978b-2218-4013-a10a-b7e3f8d086b6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.572662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.573607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.573977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.574418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.582417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.584088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:45.599661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:45.600809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:45.601841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:45.605241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 340/692] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:45.606369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.627055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55695bea-0ead-49e6-8ab9-80f8f15c7008 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.664375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.665562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.666150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.666607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.678864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:45.687563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:45.688561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:45.689544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:45.692433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 353/693] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:45.693501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.719248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f4fa-4b1c-45b6-9bb3-ca2642b6aeac None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.758256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.759233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.760358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.760842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.761335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.771833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.773559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:45.792029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:45.792974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:45.794499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:45.794499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:45.794499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:45.794499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:45.796763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 341/694] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:45.797911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.808843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:45.815389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fede886-a473-4e07-919b-15930bf70a49 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.854158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.855066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.855558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.856580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.857594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:45.858088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.859664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.873997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 354/695] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:45.875091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:45.915447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1ebf-3f21-4258-bab6-ccf903684963 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:45.954422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:45.955227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:45.955608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:45.955984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:45.956857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:45.957304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:45.958759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:45.972615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 342/696] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:45.974183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.014053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133b0d4-23dc-44ac-85db-4e2feb760164 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.031741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:46.048876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:46.053946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.054616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.054903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.055183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.055916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:46.056229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.057641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.077018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 355/697] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:46.078042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.086038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5eddf-1d60-4757-abb3-5172662f60b0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.124177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.125244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.125801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.126278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.127401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:46.127918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.129642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.141730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 343/698] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:45 2022] GET /identity/v3/role_assignments?user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:46.142921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.187878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d677e-e471-4000-b270-992fac41e0fb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.224775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.225615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.225901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.226179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.238672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.240090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:46.259630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:46.260620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:46.262001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:46.262001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:46.262001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:46.262001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:46.262001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 356/699] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:46.269068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.316838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:46.356515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e006221-78d6-4a35-80ae-9f3add222946 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.402682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.403913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.404204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.404635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.405611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.407058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.416735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:46.456733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de50fd-d092-4159-b0b2-5dbacd718126 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.484515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 344/700] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:30:46 2022] POST /identity/v3/users => generated 254 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:46.500445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.501314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.501651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.501931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.502673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:46.502987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.504387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.518390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 357/701] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:46.523991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.663017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78853b-27b2-4d53-a19d-e8c77cb4fdef None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.703610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.704425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.704712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.704988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.714564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.715708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:46.731043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:46.735351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:46.736287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 345/702] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:46.742194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.838716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:46.851347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4817e05-46a0-4f80-9f93-4a2eca788809 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:46.887152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:46.888422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:46.888666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:46.888851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:46.889710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:46.890018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:46.891518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949c49-5298-424f-8d8b-18eee9c6ac6b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:46.911255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 358/703] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:46.916926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:46.979055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.036427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38ffcb-cbe5-475d-90a6-821b2349e8a7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.064367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94071be3-9695-4403-8be2-eb29f54691bc None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.064677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94071be3-9695-4403-8be2-eb29f54691bc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.064931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94071be3-9695-4403-8be2-eb29f54691bc None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.066489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 359/704] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:30:47.072401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.072956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.073354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.078458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.079331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.079761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.080182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.092226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:47.093801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:47.202418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.203878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.222463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 360/705] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:47.252260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.285392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.378088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 346/706] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 12:30:47.384420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.397415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.442823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e0ed6-f079-4949-a73e-0b2a9332c83b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.514737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784b1c9-bdfa-4d10-a204-02bbb6131e01 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.518310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.519144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.519426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.519696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.520356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.521756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:47.536411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:47.537464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:47.538749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 361/707] 10.222.0.47 () {56 vars in 1129 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:47.543794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.574759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.576329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.576610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.576882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.591296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.593484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-41625d4c-999f-4903-b234-21a1bec26702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:47.617496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:47.618544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:47.620040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:47.620040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:47.620040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:47.620040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:47.620040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 347/708] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:47.623314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0b5e-45c8-4a35-bdbc-a0425e56e8fb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.623314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0b5e-45c8-4a35-bdbc-a0425e56e8fb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.623314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0b5e-45c8-4a35-bdbc-a0425e56e8fb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.624082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 348/709] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:47.624568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0b5e-45c8-4a35-bdbc-a0425e56e8fb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.629339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.669455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf29-671c-4d8b-9416-959e9d4de473 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.709872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.710793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.711280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.711812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.712760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:47.713242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.714811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971890c-dd8f-43c2-8cec-00a7c5b05277 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.721503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 362/710] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:47.723142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.723646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.724128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.734183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.747422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:47.748982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:47.757632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0b5e-45c8-4a35-bdbc-a0425e56e8fb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.794397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.795097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.795378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.795653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.796384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:47.796686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.798193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.811342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 349/711] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:47.812185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.833058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 363/712] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:47.834243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:47.878671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:47.920933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167491e-da97-4327-b8ce-1b0feecd2401 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.956777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:47.957473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:47.957756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:47.958030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:47.958835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:47.959144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:47.960673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:47.962890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e773ab-8533-4690-893f-aa2572eaf30f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:47.967309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 350/713] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:47.968358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:48.012269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.013707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.015314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.017421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.029421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:48.032950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.037578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:48.067786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:48.068920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:48.070172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:48.074159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 364/714] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:48.079437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:48.196936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15badb5-211d-4659-930b-e00cde5134f8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:48.225795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:48.235917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.236721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.237010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.238084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.238659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.240241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.243504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773db3a-719c-4c80-aaf0-573330382f93 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:48.255401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:48.260651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:48.261750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 351/715] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:47 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:48.263513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:48.287666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.288397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.288687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.288978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.289842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:48.290252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.291857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.305682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 365/716] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:48.314194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:48.369843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5c5a7-a7d0-4841-9b1b-a64c5fff1b75 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:48.417457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:48.425151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.425827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.426104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.426376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.427090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:48.427396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.429545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.439748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 352/717] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:48.446978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:48.532951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:48.535136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfbf61-4fa8-442a-a874-5754642ceee8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:48.558358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68792040-e2e3-40a3-8c97-3224e54f71b4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:48.595597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.596273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.596554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.596833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.597590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] RBAC: Adding query filter params (type=network) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:48.597892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.599485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9062e8-28b0-493b-afc2-7588fca9952a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.604556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 353/718] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:48.632670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:48.633384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.633666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.633959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.634800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:48.635113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:48.636601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b1554-9335-452b-bca2-99f3cab4633e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:48.650426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 366/719] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:48 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:48.907456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3f7d-4f18-4c93-8b7a-5cce8ed57dcf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.907833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3f7d-4f18-4c93-8b7a-5cce8ed57dcf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.908023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e3f7d-4f18-4c93-8b7a-5cce8ed57dcf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.909614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 354/720] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:48.915153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:48.915441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:48.915717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:48.934636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:48.936150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:49.012121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 367/721] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:49.035567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3f7d-4f18-4c93-8b7a-5cce8ed57dcf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.144089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e3f7d-4f18-4c93-8b7a-5cce8ed57dcf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.165497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:49.189226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:49.190044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.190314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.190589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.202590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-16c18fc9-c912-43cf-bc88-28b6a5f3c546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:49.210354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:49.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:49.213761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 355/722] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:49.219334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.269902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:49.323365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8011a-bfb9-4345-8d17-903e45b8eeed None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.334719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08c8d-eea3-428d-9d0e-a4a852165e05 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.335091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08c8d-eea3-428d-9d0e-a4a852165e05 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.335441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08c8d-eea3-428d-9d0e-a4a852165e05 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.338145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 356/723] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:30:49.343059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.343417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.343760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.362367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:49.363277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.363706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.364150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.365073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:49.365523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:49.367331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:49.377161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 368/724] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:49.381150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:49.381150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:49.384274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.503581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 357/725] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 12:30:49.525674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.587642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bfee3-3e89-443b-ac02-c61e34d359ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.609485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:49.635317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556936-1811-4046-83a3-f834492b261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:49.636148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556936-1811-4046-83a3-f834492b261a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.636435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556936-1811-4046-83a3-f834492b261a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.636711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58556936-1811-4046-83a3-f834492b261a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.647717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58556936-1811-4046-83a3-f834492b261a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:49.649267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58556936-1811-4046-83a3-f834492b261a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-58556936-1811-4046-83a3-f834492b261a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:49.679325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:49.680551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:49.681538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:49.681538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:49.681538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:49.681538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:49.682926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 369/726] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:49.688945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556936-1811-4046-83a3-f834492b261a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.707748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582552d7-8bdf-4cbc-a6d6-410d3b83a61a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.719886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:49.769573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:49.770288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.770618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.770894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.789903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:49.797784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:49.798926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:49.799808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:49.802045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 358/727] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:49.810131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:49.912950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc28d5b-a59c-434f-95f3-82adf6cde771 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.925558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58556936-1811-4046-83a3-f834492b261a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:49.948153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:49.949077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:49.949530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:49.949949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:49.950863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:49.951313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:49.952810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:49.961490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 359/728] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:49.968479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.004968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.007036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.008209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.008630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.009531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:50.012672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.014230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.037528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.038308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 370/729] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:50.044154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.090655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.186794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a3ae3-31e7-42fb-aefd-6e6e5f0d099b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.239647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8e728-9352-4733-a309-1a39556cbad2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.252807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.253636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.253915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.254187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.273810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.275254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:50.305143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:50.306118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:50.307612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:50.307612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:50.307612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:50.307612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:50.307612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 360/730] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:50.312310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.313183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.314275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.314814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.315289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.323431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.325276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:50.349445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:50.350437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:50.351397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:50.358177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 371/731] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:50.363795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.482274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.501369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fee81-059d-4e7f-a3f2-6a93e40fffbe None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.503371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.535595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8878ae1-5d43-4b16-831f-4d2bd0432450 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.538233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.539489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.539489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.539722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.540760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:50.541214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.542758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5515f4-7c16-411a-82ed-2886126db16e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.563512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 372/732] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:50.565214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b7069-487f-4f31-a15a-6fc41e8fb67a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.565848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b7069-487f-4f31-a15a-6fc41e8fb67a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.565975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b7069-487f-4f31-a15a-6fc41e8fb67a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.571445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 373/733] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:50.571445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b7069-487f-4f31-a15a-6fc41e8fb67a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.579201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.585182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfe678-aeb2-4e7e-b229-a9191f88c9cb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.601287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 361/734] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:50.603033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.603467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.603885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.624423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:50.625973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:50.678084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b7069-487f-4f31-a15a-6fc41e8fb67a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.705392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 362/735] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:50.706796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.720755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.721488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.721829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.722106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.738474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.798914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.800356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.800724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:50.834646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc3a2f-a6a7-4ef3-b5df-d16b065fc1af None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:50.873905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:50.875117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:50.875598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:50.876065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:50.884205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:50.886021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:50.903219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:50.904262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:30:50.905228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:50.905774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 374/736] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:50.905774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.912544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 363/737] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:50.913703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:50.948905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.009148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d12a57-0673-4312-803a-ccb18dd328fa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.024240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1daed-6b8a-4687-a2d8-a89b10e8e12f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.045084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.045757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.046036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.046311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.047030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.048387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543be58-35e6-42f0-ae5a-432703ec0f96 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.056280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.061274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.062066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.062349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.062683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.072762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.074286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:51.092539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:51.093562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 364/738] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:51.094505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.159481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 375/739] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:30:50 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:51.160926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b679-da9a-4216-af23-5f647f0f83dd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.161260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b679-da9a-4216-af23-5f647f0f83dd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.161665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b679-da9a-4216-af23-5f647f0f83dd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.162925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 376/740] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:51.163761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b679-da9a-4216-af23-5f647f0f83dd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.218929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec64fd4-3140-45c4-8301-483d437ebbc1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.277998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.278692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.279175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.279461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.280337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:51.280625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.280739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.282196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bab7fc-df3a-4471-bb77-909909b00f59 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.284881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b679-da9a-4216-af23-5f647f0f83dd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 365/741] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16eefb-0e48-4630-890d-ec99c71f655e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16eefb-0e48-4630-890d-ec99c71f655e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16eefb-0e48-4630-890d-ec99c71f655e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 366/742] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:51.307195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe40a-0858-43b5-b2a8-554ad32cb81d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.307739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe40a-0858-43b5-b2a8-554ad32cb81d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.308154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe40a-0858-43b5-b2a8-554ad32cb81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.328494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d77fe40a-0858-43b5-b2a8-554ad32cb81d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:51.329948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d77fe40a-0858-43b5-b2a8-554ad32cb81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:51.334225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.334620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.335768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.335932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:51.336192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.337716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.354198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 377/743] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:51.354643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.364688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.407196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 367/744] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:51.408727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.409136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.409487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.426289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:51.427590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:51.479344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ee1d-2409-4e5a-b3ac-9aed72c062da None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.503887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 368/745] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 12:30:51.503887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.515673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.516524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.517071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.517426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.551061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.599601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.601418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.630290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.635671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e41bcb-f9fc-46d0-b733-0d30fb4a428f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.687207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.688018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.690628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.690628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.696076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.697912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:51.715927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:51.716909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 369/746] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:51.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.731425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 378/747] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 12:30:51.732422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.798261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:51.838184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224409-cf47-42f4-948c-8b859b42ed29 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.847235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd515f-b304-4f0e-aa99-7dee9c19d84f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:51.877456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.878277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.878562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.878834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.879683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:51.881112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:51.886098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:51.887521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:51.887806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:51.888258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:51.899050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:51.907353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:51.908459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 379/748] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:51.920886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.961379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 370/749] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:30:51 2022] POST /identity/v3/services => generated 240 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:51.961379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:51.985592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.072603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef35a-0bc7-4470-ae66-d7421642eeb2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.098232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.122030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff31a-833e-413e-b77f-13d84fe2b803 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.127852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.128889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.129443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.129914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.130943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:52.131516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.133357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.154058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 380/750] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:52.155257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.175318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.178169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.178536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.179050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.203010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.204302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:52.230185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:52.231194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:52.232123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:52.232123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:52.232123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:52.232123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:52.232123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 371/751] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:52.236168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.309814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fb73-9597-4572-b778-1d9810bbbf13 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.316114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.360470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.360853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f249a5-9cc6-43dc-8e62-6bbe32deecb6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.361615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.362058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.362573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.365325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:52.369479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.371787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.385016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 381/752] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:52.386114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.405519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.406240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.406521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.406796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.407846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:52.408535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.410370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.417538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 372/753] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/role_assignments?user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:52.423070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.441938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.508975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8538c-14a8-4258-a862-c700e5cf1290 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.546744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.547409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.547692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.547967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.548860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:52.548971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.550388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.567359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 382/754] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:52.569259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae00e5-7ece-4815-b285-b52401e792e5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.576448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.604828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.622613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.623770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.624196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.624632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.649717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.652073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:52.682535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:52.684141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:52.689353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:52.689353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:52.689353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:52.689353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:52.690752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 373/755] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:52.697567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.705148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee34c7-864f-4f1e-9108-2df67ccb33d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.709392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.741131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.742001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.742287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.744409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.750507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.753031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:52.768229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:52.769240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:52.770180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 383/756] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:52.775262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.877988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f00e4-07d3-4d2d-858b-95e4958c4f4d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.913507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.914084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.914361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:52.914632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:52.915472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:52.915771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:52.917370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:52.927805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad583833-5f5a-4f0c-8d5c-6a79f7959a4f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:52.932052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 384/757] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:52.936269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:52.979562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:52.996155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:52.997165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:52.997619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.002263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.002263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:53.002263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.002263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.015696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 374/758] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:53.023437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.089189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788d47-bb11-49ae-a4b3-eb8a79a3bc70 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:53.105403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:53.162505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:53.163156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.165415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.165415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.165415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d1051f87dc384cfe86485b5e290982e2) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:53.165750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.167148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.187026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 385/759] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:30:52 2022] GET /identity/v3/role_assignments?user.id=d1051f87dc384cfe86485b5e290982e2&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:53.203416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331306-77c6-4e9d-87d8-c7c464a89cfb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:53.265908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:53.266780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.267059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.267328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.275597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.278155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:53.294248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:53.295286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:53.296254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:53.299316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 375/760] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:53.305430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.407396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:53.432887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7cab9-23df-4a27-9500-f8900d10adc7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:53.469630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:53.470304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.470602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.470900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.471739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:53.472051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.473526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe19ede-4bfd-4594-bf24-61eaf1beae53 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.491416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 386/761] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:53.497384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.618739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b954-3c85-4bf7-915f-9aa5717ed61a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:53.620342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:53.656288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:53.657111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.657398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.657676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.668676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.670292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.687750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3f0a6-4ef4-4471-b24e-9cfb8ebc4bc3 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.687750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3f0a6-4ef4-4471-b24e-9cfb8ebc4bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.687750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3f0a6-4ef4-4471-b24e-9cfb8ebc4bc3 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.687750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 387/762] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:53.691501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:53.692400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:53.693356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:53.693356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:53.693356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:53.693356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:53.693594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.693594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.693594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.695616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 376/763] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:53.704973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.719218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:53.720039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:53.843711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 388/764] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:30:53.875541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa632c00-be00-41db-a7b1-783a9f4d1601 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:53.898702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.943838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:53.944436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:53.944722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:53.945002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:53.945778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:53.946085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:53.947768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:53.960879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 377/765] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:53.967504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:53.997469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.067886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.080793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e22d-6f8d-4c9a-b8b8-fa32aec9d892 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.109852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738745a1-1b47-474a-92c9-e2b0c46a1b0e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.141915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.143108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.143608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.144100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.159407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.161891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.174451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.182029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.189452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.189740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.190438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.191833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:54.196032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:54.196921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:54.197865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:54.201696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 378/766] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:54.207718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:54.218278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:54.219228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:54.220499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 389/767] 10.222.0.47 () {56 vars in 1129 bytes} [Sat Jul 16 12:30:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:54.225340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.348245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.375103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc663b-0cf7-459c-913c-683a3fc224af None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.382324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef157e65-05b7-4a84-bdb9-1c572c8aba1c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.415307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.416221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.416499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.416771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.418730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:54.419099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.420945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.424042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.424784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.425157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.425497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.426273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:54.426647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.428178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e8152-c60b-4bb9-9d57-eee56174abd3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.429253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.435201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 390/768] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:54.436615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27fc0-c121-4b49-ba65-58272ac54c2f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.437065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27fc0-c121-4b49-ba65-58272ac54c2f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.437508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27fc0-c121-4b49-ba65-58272ac54c2f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.442874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 379/770] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:54.442874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.443170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 391/770] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:54.447961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.447961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.447961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.463655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:54.465368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:54.553061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 392/771] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:30:54.553061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.590038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc05b40-8a94-4bab-aa95-ddb3ba937bad None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.630055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.631078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.631548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.632016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.632912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:54.633439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.635144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.638240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.648735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 380/772] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:54.654697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.692737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083f372-8ded-43e5-bd33-0a823f3819fd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.724984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.733872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.738676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.738676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.738676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.738676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=0faa6a54416f420bbdec92832d404efd, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.739103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.788600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc7dbd-d69c-4802-85ca-566228e666db None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.820913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 393/773] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:54 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:54.820913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.846744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:54.847565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.847890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.848165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.848825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:54.850239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e1cb2db3-dd7e-4c03-b4e0-c68928b25a62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:54.869490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:54.870593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:54.871875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 381/774] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:54.872757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc492-1e32-4d1a-973b-35711239f96a None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:54.872757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc492-1e32-4d1a-973b-35711239f96a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:54.872757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc492-1e32-4d1a-973b-35711239f96a None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:54.878222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 382/775] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:54.879069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc492-1e32-4d1a-973b-35711239f96a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:54.947539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:54.978082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83b2ca-e7ab-4342-8ed5-436b2f59bb7e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:54.992877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc492-1e32-4d1a-973b-35711239f96a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.023256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.023924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.024329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.024610 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.025470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] RBAC: Adding query filter params (interface=public, service_id=67671997084d4f92a55b779dfb8bab1e, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:55.025788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.027749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f5b7-c328-4a6b-9de3-3be80c84f4a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.033724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.034443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.034769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.035163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.035875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] RBAC: Adding query filter params (name=network) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:55.036180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.036812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 394/776] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/endpoints?service_id=67671997084d4f92a55b779dfb8bab1e&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.037736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.038245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.038543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.039111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.047556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 383/777] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.048221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.066185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:55.067390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:55.116710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.137083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 395/778] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:30:55.142823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.171114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d4ab-9148-46bb-bab4-f844132aa475 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.201100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.210725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.211657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.212093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.212552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.213423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.215762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.222111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 384/779] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.229149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.253428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be3dae-de6f-46c9-86bf-6e6dae01ebfb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.295022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.295961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.296319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.296667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.298035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] RBAC: Adding query filter params (type=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:55.298035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.299540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.307148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 396/780] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.313445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.368202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.374369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c08e-a93c-458b-9f08-95ebc1587a69 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.414225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.415354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.415852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.416327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.428634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:55.436971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:55.438044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:55.438903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:55.441619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 385/781] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:55.447556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.476172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf856-908e-46e9-a7a6-fe1f19abe6ea None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.526551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.531481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.532628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.533246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.534094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.535385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] RBAC: Adding query filter params (interface=public, service_id=78630b48c09443f191637130e63b1dd0, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:55.536059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.538307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.546374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 397/782] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/endpoints?service_id=78630b48c09443f191637130e63b1dd0&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:30:55.551779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.599273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455427-5192-4113-9a8a-ef8a421fc6c9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.670233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.671065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.671409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.671752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.672577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:55.673048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.674427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.689329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 386/783] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.695022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.723714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655fc9f-4294-4d15-8d16-0ae92fc94961 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.737359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.771040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.772237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.772621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.773096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.773883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.776861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8b1e6-46c2-4c6d-b800-89d3b499b9f5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:55.782156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 398/784] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:55.784046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45e71-f3cb-436d-8c02-67a2b62a4111 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.784339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45e71-f3cb-436d-8c02-67a2b62a4111 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.784619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45e71-f3cb-436d-8c02-67a2b62a4111 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.786187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 399/785] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:55.799260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.799602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.799859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.834840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:55.836090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:55.909193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e22f9-559b-40e4-b9ea-cbdb5a2033a4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:55.938965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:55.958534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 400/786] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:30:55.971482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:55.972280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:55.972607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:55.972970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:55.982979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:55.992288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:55.993837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:56.014872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:56.015827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:56.016711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:56.016711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:56.016711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.016711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:56.016711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 387/787] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:56.023458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.106528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fab8be-8459-4183-84b1-5460b092def7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.108336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.141882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.142656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.142944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.143221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.155583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f368a7-ea2e-4220-b12e-1e68d420bb13 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.157508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:56.165363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:56.166337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:56.167203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 401/788] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:30:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:56.182994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.200274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.201358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.201846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.202317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.203275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:56.203783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.205436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.219291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 388/789] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:56.238494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.273404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.323289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdf55-ed34-4e98-9f6c-2dbb0a09ef20 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.354068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700bb65-e9f7-446e-b065-3e60b620dcd5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.389669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.390572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.391082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.391267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.391531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.391812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.392158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.392563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:56.392843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.392954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.394476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.401133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.403063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.410521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 402/790] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:56.415671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:56.425257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:56.426243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:30:56.427202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:56.429856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 389/791] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:56.439954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.454246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.530393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd90ef-05bc-4b32-856d-bb8b92a34cb0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.569823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.571107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.571600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.572332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.584755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.586807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:56.607276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:56.608245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:56.609198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:56.609198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:56.609198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.609198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:56.611594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 403/792] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:56.616742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.633606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.644793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029798e-18e6-47ca-9e49-631eb1d98b54 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.690064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.690745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.691024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.691299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.692148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:56.692452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.693916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.725802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 390/793] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:56.732178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.779176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.827981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15de8-0d6f-404d-9b70-add71b69fab2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.871683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a9755-b17a-4144-bb52-c675db655728 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:56.881987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.882654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.882938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.883215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.883942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:56.884254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.885653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.897722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 404/794] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:56.903113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:56.939975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:56.941485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:56.941966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:56.942457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:56.953648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:56.955358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:56.973626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:56.974567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:56.975464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:56.975464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:56.975464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:56.975464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:56.977876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 391/795] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:56.978058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:56.984102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.076297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11546d4-50a9-4d17-8df7-b797c2637881 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.112840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.113690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.113974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.114249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.122674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.124139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.125619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ec02c-d6e7-477e-a098-6b967b3b6351 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.126112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:57.139565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:57.141641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:57.142579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 405/796] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:57.146728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.168996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.170046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.170561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.171088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.172040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:57.172542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.174560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.189450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 392/797] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:57.194818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.269695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc24c12-2aea-4c97-9bbc-4a0dad44a25c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.274190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.313725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.314401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.314680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.314951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.315781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:57.316083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.318161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75dc7-fa2a-43ad-a595-ab7ab019ba6e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.335147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad940bf-00e3-458f-8703-74f060163221 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.376370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.377295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.377619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.377892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.386111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.388505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1da11ade-05d6-44b5-8ace-ca89989d00f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:57.403860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:57.404828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:57.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 393/798] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:57.411690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16339b8-78a6-461f-918e-b6ef341f8386 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.411989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16339b8-78a6-461f-918e-b6ef341f8386 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.412252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16339b8-78a6-461f-918e-b6ef341f8386 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.414357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 394/799] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:57.415419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16339b8-78a6-461f-918e-b6ef341f8386 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.473855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 406/800] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:57.475281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.475577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.475855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.493425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:57.494733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:57.539730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16339b8-78a6-461f-918e-b6ef341f8386 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.549549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.556619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.573396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 407/801] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:30:57.574685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.579802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.580723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.581259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.581752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.582737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:57.583413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.585075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.601431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 395/802] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:57.607999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.699386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a4ce0-ea47-4d83-94d1-c4bd3150d5dc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.715966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982e6e-2e6a-41b0-a897-02014bb8ac58 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.736327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.737158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.737437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.737695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.748449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.749909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.753404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.754191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.754471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.754738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.766754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:57.768003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:57.768894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 408/803] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:57.774136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f8aa040f-4db3-4852-99c2-0d67dce174a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:30:57.774507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:57.775359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 396/804] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:57.776701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176d93-692f-42e7-893f-e5a782525112 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.778618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176d93-692f-42e7-893f-e5a782525112 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.778618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176d93-692f-42e7-893f-e5a782525112 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.779867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 397/805] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:57.780615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176d93-692f-42e7-893f-e5a782525112 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.819256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.821786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:57.885449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a17-b985-4633-9612-2164d725aa79 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.891487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176d93-692f-42e7-893f-e5a782525112 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:57.923416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.924301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.924579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.924844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/6d7e37eb8b3e44d5a7f2be9af4a02c1f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.926779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=6d7e37eb8b3e44d5a7f2be9af4a02c1f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.927190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:57.927849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.928129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.928228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdf6b3-2438-4a4c-a745-4d4a5f75a4a4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.928404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:57.929155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:57.929463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:57.930799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:57.942989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 398/806] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:57.943785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:57.997161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 409/807] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:57 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/6d7e37eb8b3e44d5a7f2be9af4a02c1f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:57.997161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:57.997161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:57.997161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.021103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:58.022436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:58.048328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.055948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7610-8adc-4945-a2a5-cd7e43ca75e8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.104368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.106256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.106847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.107270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.108313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:58.108685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.110890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.117907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 410/808] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:30:58.118799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.135186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 399/809] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:58.142911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.174406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.259058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170b3ac-95f4-480f-b6c6-8bf8adc15ce8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.272887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4654a-ca27-419b-bb36-88b34237e4f0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.301647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.302940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.303324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.303695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.313277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.314978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.315557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.316543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.317174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.317587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.329702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:30:58.331280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:58.332339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:30:58.333284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:58.333723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 411/810] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:58.334368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.334689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:30:58.357369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:58.358329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:58.359239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:30:58.359239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:30:58.359239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:30:58.359239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:58.361691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 400/811] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:30:58.362659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.376506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.432604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.445788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c843a-0335-4d83-abc4-eb36b780a1ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.473293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba9a6e-8db8-4361-a1c5-73fcc42a4f1f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.481866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.482672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.482953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.483228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.483910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.485275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-64cd72fc-20cc-44ea-9ac7-c81c9a9b6690 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:58.498135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:58.499265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:58.500230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 412/812] 10.222.0.47 () {56 vars in 1129 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:58.501432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d520a-4187-40c3-8ad4-028c7cb08b56 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.501432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d520a-4187-40c3-8ad4-028c7cb08b56 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.501432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d520a-4187-40c3-8ad4-028c7cb08b56 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.502259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 413/813] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:30:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:30:58.502943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d520a-4187-40c3-8ad4-028c7cb08b56 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.518269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.518863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.519145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.519411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.520200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:58.520508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.521958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.535164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 401/814] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:58.535972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.605927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.621484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d520a-4187-40c3-8ad4-028c7cb08b56 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.644349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9b28a-857f-41d6-b22b-4f2ccbcc4592 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.656007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.656661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.656943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.657248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.657974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:58.658302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.659632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67619cf-b7c7-467f-9437-c576330f151f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.671265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 414/815] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:58.672533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.672822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.673132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.686869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.687881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.688426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.688915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.689506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:30:58.690060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:58.690542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.690698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:30:58.692230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.697825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 402/816] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:58.698872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.704685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.766885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 415/817] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:30:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:30:58.778129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.807363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3c61f-c2cc-478f-ac48-d40caea8861b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.844673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:58.851224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.852387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.852699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.853032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d1051f87dc384cfe86485b5e290982e2/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.873710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=d1051f87dc384cfe86485b5e290982e2, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.875568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.884461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423b26-07a8-4853-8762-abfd2a2b1cf7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:58.917371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:58.918861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:58.919139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:58.919412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:58.926158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:58.927703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:30:58.942954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:58.944045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:30:58.945323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:30:58.948582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 416/818] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:58.949460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.951892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 403/819] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:30:58 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d1051f87dc384cfe86485b5e290982e2/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:30:58.951892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:58.961828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.070291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4484dd-7cd5-414f-a570-e501a434a8ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.099152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec745-6ef5-49f4-bbbf-d9a2ef41174b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.124333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.125330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.125621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.125901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.126886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:59.127638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.129310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.136656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 417/820] 10.222.0.47 () {56 vars in 1124 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:59.137616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.139851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.141324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.142669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.143149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.143991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.145592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:30:59.160895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:30:59.161972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 12:30:59.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:30:59.165396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 404/821] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:30:58 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:30:59.166303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.222589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.227574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.275194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd88e6b-de39-4885-9ff9-531aad3fa23f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.287744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7247b-c7ae-4696-8720-2ae439bd4f0f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.314867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.315778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.316209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.316630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.318245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:59.318707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.320381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.334453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.335167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.335455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.335733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.337104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.338821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.341226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 418/822] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:30:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:59.342466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.446701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5595-c9fa-4cb0-9cb1-35f30ebfe365 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.453469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 405/823] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:30:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:30:59.455741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.497094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.536223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.536908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.537214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.537487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.538209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] RBAC: Adding query filter params (name=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:59.538516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.539922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.559596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 419/824] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:59.565440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.631845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884d8f9-367b-4f3d-90db-81953c248f90 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.634139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.669942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.670880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.671309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.671742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.720614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0512a-fad6-4b5b-adce-88f988589040 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.745263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.748707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.765507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.766319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.766807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.767180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.769002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:30:59.769415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.771057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.778654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 420/825] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:30:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:30:59.784247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.861447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.862675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 406/826] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:30:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:30:59.870433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:30:59.873958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:30:59.897677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2cf7d-8ccf-4791-8073-af57078d8af0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:30:59.961759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:30:59.962442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:30:59.962806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:30:59.962994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:30:59.963755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:30:59.965190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcf404-86ac-4d77-aad8-69bb6c0679c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:30:59.979237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebacef5-4f04-4c34-999c-c0364ccf128c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.016091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.017186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.017615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.018079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.029997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.031684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:00.051020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:00.052171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:00.053296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:00.053296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:00.053296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:00.053296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:00.055199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 407/827] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:00.062099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.070687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 421/828] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:30:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:31:00.125961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:00.179252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8c57a-08d9-4e28-a0a7-2df8ad054f27 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.204445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbc25-07ff-413e-bd4d-50b59e5c51c5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.204958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbc25-07ff-413e-bd4d-50b59e5c51c5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.205056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbc25-07ff-413e-bd4d-50b59e5c51c5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.206626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 422/829] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:00.211876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.212162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.212437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.229732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.230952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.231330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.231611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.232193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:00.232362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:00.232634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.233427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:00.234025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.248140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 408/830] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:00.253758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.332880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 423/831] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:00.373525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.396466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006956-e81a-4e45-bb5c-1bde0d0f7221 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.407478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:00.422440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:00.435344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.435969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.436327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.436657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.444674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.446249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:00.463198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:00.469927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:00.470842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 409/832] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:00.471763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.508699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36454a7-39f4-4c07-8f03-2ec19dbbb3aa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.546163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.547078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.547360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.547628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.559443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:00.571460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:00.572432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:00.573331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:00.574616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 424/833] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:00.582215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.588757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0dc71-3c7b-41cd-bd0c-09160cccfe30 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.631307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.631973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.632254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.632526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.633347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:00.633652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.635095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.649213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 410/834] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:00.656255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.698834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:00.711277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:00.719761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2434c-62e5-4581-a092-3449d5d52a64 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.756376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.757813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.758597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.759094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.760030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:00.760515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.762857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.778265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0bf5-a7aa-4905-a746-97d9419e4c7e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.778604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 425/835] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:00.779663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:00.811745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.812596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.812931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.813390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.814216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:00.814683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.816217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.822925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 411/836] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:00.894668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e543c3-3d18-4960-8547-5257f18b8b01 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:00.933136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:00.934198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:00.934657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:00.935083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:00.947496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:00.950016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:00.969741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:00.970899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:00.972015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:00.972015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:00.972015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:00.972015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:00.973939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 426/837] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:00.979644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.079784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.086739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78798b48-573d-4c1f-998b-c044eed3e6f5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.121623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.122524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.122951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.123372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.124243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:01.124718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.126242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003637d7-6806-486e-a3c1-f8c28342affc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.139387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 412/838] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:01.144976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.158148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.250846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd5386-5587-418c-b711-8d9cc5a48734 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.286692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.287487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.287775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.288050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.297180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.298553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.311781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96482fc-b73d-45c6-b3fb-79ffdf7636cd None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.312066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96482fc-b73d-45c6-b3fb-79ffdf7636cd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.312342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96482fc-b73d-45c6-b3fb-79ffdf7636cd None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.313973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 413/839] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:01.316454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:01.317440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:01.318393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 427/840] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:01.323388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.323561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.324215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.326160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.342091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:01.343311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:01.402874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.433740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 414/841] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:01.446065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e88c-4fe9-446a-84aa-870768a9c679 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.456351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.493466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.494862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.496182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.497114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.498450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:01.498969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.501752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.525751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 428/842] 10.222.0.47 () {56 vars in 1135 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:01.531557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.563263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.582396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792fbfd-8a5e-4066-beb2-1080798c1c31 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.622655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.623919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.624415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.624950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.637402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:01.637665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a2227-8c6f-472a-addc-d01bea89394e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:01.646005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:01.647139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:31:01.648201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:01.655824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 415/843] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:01.657256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.664317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.674375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.675180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.675466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.675736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.746244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.748032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.773396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249b91e-455c-43fe-9483-ba304c16bd91 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.811617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.812587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.813021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.813545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.814438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:01.814944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.816581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.830912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 416/844] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:01.832502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.842617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.851447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 429/845] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:01.858154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:01.885841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:01.943642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea922-58a8-4b72-884e-8d6b68bfcc14 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.961795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8193e-f164-49e5-b37e-ac84e3a67870 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:01.979926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:01.980769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:01.981077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:01.981364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:01.991768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:01.993232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:01.999063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.000347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.000629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.000899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0a301cc8-d806-4bbb-b496-86d75daf9978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:02.010495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:02.011960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:02.012852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:02.012852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:02.012852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:02.012852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:02.014632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.016078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.017022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 417/846] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:02.018363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0611b-45bf-41a8-b06a-cf901074188d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.018660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0611b-45bf-41a8-b06a-cf901074188d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.018947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0611b-45bf-41a8-b06a-cf901074188d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.020249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 418/847] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:02.022254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0611b-45bf-41a8-b06a-cf901074188d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3ba70528-9573-4d09-bb3e-6218fdf446a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:02.038744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:02.039726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 430/848] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.046195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.060377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:02.061628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:02.094845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.145065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 431/849] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:02.145065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.146985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0611b-45bf-41a8-b06a-cf901074188d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.157983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.183560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.184247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.184523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.184797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.185564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.185871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.187235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.199800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 419/850] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.200630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.261661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa9fea-fa8d-49ae-a44b-d88ca17e97b6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.298172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.298764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.299048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.299322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.300043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.300346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.301740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.307121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819b5a5-9efe-44c6-b463-39214efb3ebf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.313366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 432/851] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.314166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.328772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.348342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.349299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.349583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.349855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.350503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.352368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:02.364933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:02.366127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:02.366970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:02.375192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 420/852] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:02.376270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.384743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.419748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6d80-edf4-4acc-ae74-d090414629e2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.455664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.456576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.456918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.457305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.467260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.469006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:02.484190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:02.485404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 433/853] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:02.486704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.494787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b127-b1c2-4c25-be2c-1062192773f3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.527575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.528365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.528640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.528910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.535705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.537166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:02.552643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:02.553790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:02.554975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:02.558490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 421/854] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:02.559281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.604747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.609963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae2498-fab1-484f-889f-43ce3b6bc83b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.637565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.645533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.646187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.646463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.646730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.647524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.647978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.649358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.659054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 434/855] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.659054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.670938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ad75-4067-48f4-a9c8-7040a8de76e2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.710025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.710781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.711136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.711498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.712408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.712800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.715019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.733701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 422/856] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.734522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.768428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760444-9df8-4ab3-a5ef-ded7df152268 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.806782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.807356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.807630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.808557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.809640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.809640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.811086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.823835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 435/857] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.847187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.850563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decf5cb-e16b-4fa8-8304-7fd233f2d6a9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.882572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:02.887039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.887725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.887992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.888270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:02.889001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:02.889335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:02.890701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:02.905339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 423/858] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:02.905339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:02.946991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369538c1-c1ae-4d8b-b764-a4e2249fb452 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:02.988789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:02.989464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:02.989742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:02.990124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.028636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96171566-7de5-4b10-b238-52c2b777cc06 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.065954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.066531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.066652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.066815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.067094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.068621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:03.068730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.068822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.069809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7328d-4f33-450f-a31e-a35c83359f71 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.078534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 424/859] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/role_assignments?user.id=6d7e37eb8b3e44d5a7f2be9af4a02c1f&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:03.080613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cb058-7d20-49de-ba4d-43411789ff4e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.080613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cb058-7d20-49de-ba4d-43411789ff4e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.080613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cb058-7d20-49de-ba4d-43411789ff4e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.081950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 425/860] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:03.087368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.087727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.088013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.114232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:03.116420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:03.145432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.150967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.209347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 426/861] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:03.212136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 436/862] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:03.218657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.233106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.340167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893f17-fc56-4749-a22f-4c53d30d1e02 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.362467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbbbf3-08f1-4efd-ad5d-4ea5f7cc575e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.391068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.392239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.392597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.392936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.405339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.406776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.409718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.410790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.411267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.411681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.412485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.414037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:03.426736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:03.427847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:03.428960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:03.428960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:03.428960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:03.428960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:03.428960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 427/863] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:03.434027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:03.434974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:03.435893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:03.436765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:03.443786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 437/864] 10.222.0.47 () {56 vars in 1132 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:03.443786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.452553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.481573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.555489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c577737-cb3b-4179-b441-209346ee91c5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.596140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.597227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.597772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.598197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.599065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:03.599519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.601088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.611012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 438/865] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:03.612078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.674118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2462b15-3384-4341-86a4-572d0dc1e81a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.709965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.724166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.724934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.725249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.725525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.726415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:03.726729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.728678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.731241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1906fc-d5f0-4d65-853c-34b2ee143b81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.748361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 428/866] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:03.749276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:03.780271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.781558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.782529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.783276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.784206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:03.784733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.787395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.794580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 439/867] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:03.830045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:03.866151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305251b-a65c-4295-8c07-60a999e3c9a7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:03.907080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:03.907894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:03.908176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:03.908451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:03.916646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:03.918793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:03.934249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:03.935242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:03.936593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 429/868] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:03.942168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:04.050717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f46a-6903-4298-a44c-178d3f93ddab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:04.087231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:04.087906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:04.088188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:04.088467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:04.090376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:04.090910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:04.093523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0897-1d7e-4f61-af6e-e9f91b45b751 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:04.106692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 440/869] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:04.112710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:04.222196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd498d2e-e15b-49e0-9c91-c62ea6ad1784 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:04.251795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:04.258124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:04.258825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:04.259112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:04.259392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:04.260285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d1051f87dc384cfe86485b5e290982e2) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:04.263720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:04.263720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e84d62-3a93-4964-ba58-cd67a0d608f3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:04.269139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 430/870] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:04 2022] GET /identity/v3/role_assignments?user.id=d1051f87dc384cfe86485b5e290982e2&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:05.020618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba892f6-6a09-4c35-8f54-fb5e3b6bb941 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.021639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba892f6-6a09-4c35-8f54-fb5e3b6bb941 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.022144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba892f6-6a09-4c35-8f54-fb5e3b6bb941 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.024004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 441/871] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:05.029617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda787-c788-44f6-b759-835f56e071ff None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.030177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda787-c788-44f6-b759-835f56e071ff None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.030663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda787-c788-44f6-b759-835f56e071ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.048832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bacda787-c788-44f6-b759-835f56e071ff None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:05.050426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bacda787-c788-44f6-b759-835f56e071ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:05.123269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 431/872] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:05.145896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba892f6-6a09-4c35-8f54-fb5e3b6bb941 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:05.150098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:05.220347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c465b-df8b-4844-b80d-a2d009516b4f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.220657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c465b-df8b-4844-b80d-a2d009516b4f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.220908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c465b-df8b-4844-b80d-a2d009516b4f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.222439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 432/873] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:05.227257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.227547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.227828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.246011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:05.247196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:05.269017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba892f6-6a09-4c35-8f54-fb5e3b6bb941 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:05.306519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:05.307100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.307375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.307694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.308414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:05.309998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:05.319124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 433/874] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:05.338354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:05.377578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 442/875] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:05 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:05.398444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:05.448558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7448b-4921-435b-a5e7-2b29121af81a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:05.495292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:05.496122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.496503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.496783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.497501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:05.498933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:05.512610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:05.513751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:05.515060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 434/876] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:05.520125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:05.592715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:05.633331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce3433-18cd-4e46-8de4-bab790de758e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:05.668970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:05.669943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.670370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.670787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.671641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:05.672178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:05.673739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d393d0-24be-4b79-b384-3a574fe75221 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:05.678879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 443/877] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:05.684845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:05.788574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba056bd-394d-4672-9691-03cce85e4e71 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:05.802941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:05.823936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:05.824854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.825166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.825441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.826153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:05.826458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:05.827822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02803-4e06-41b3-8a95-8f5c4d4127f5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:05.832872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 435/878] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:05.894292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d224b-1346-4988-91d1-29d898b505f5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.894569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d224b-1346-4988-91d1-29d898b505f5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.894851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d224b-1346-4988-91d1-29d898b505f5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.896602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 444/879] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:05.903065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7a87-587b-456f-9ba0-4aeaccc50dc3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:05.903374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7a87-587b-456f-9ba0-4aeaccc50dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:05.903618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7a87-587b-456f-9ba0-4aeaccc50dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:05.922153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fada7a87-587b-456f-9ba0-4aeaccc50dc3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:05.923381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fada7a87-587b-456f-9ba0-4aeaccc50dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:06.000706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 436/880] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:06.023152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d224b-1346-4988-91d1-29d898b505f5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:06.132156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d224b-1346-4988-91d1-29d898b505f5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:06.167978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:06.168617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:06.168903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:06.172098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:06.172098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:06.172098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7918117-0d1b-4835-b78d-d0360fab15a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:06.175276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:06.263049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 445/881] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:06 2022] POST /identity/v3/services => generated 269 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:06.459260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a840b7-29ec-4489-8f27-d88773802c55 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:06.459739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a840b7-29ec-4489-8f27-d88773802c55 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:06.459739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a840b7-29ec-4489-8f27-d88773802c55 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:06.461356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 437/882] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:06.467155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:06.467441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:06.467716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:06.485813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:06.486997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:06.564283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 446/883] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:06.587493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a840b7-29ec-4489-8f27-d88773802c55 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:06.702600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a840b7-29ec-4489-8f27-d88773802c55 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:06.712137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:06.737352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:06.738152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:06.738427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:06.738694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:06.749413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:06.757833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:06.758927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:06.760013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 438/884] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:06.766272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:06.874038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f5883-4417-47ca-a92d-f5af3961ddd3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:06.889289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:06.909831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:06.910499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:06.910779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:06.911053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:06.911794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:06.912096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:06.913552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:06.922428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 447/885] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:06.928055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.039313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276aad2-a30a-4234-a8c0-b48c8c1d6c42 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.074431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.075623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.075903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.076176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.087270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.088696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:07.107597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:07.108523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:07.109617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:07.109617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:07.109617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:07.109617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:07.109617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 439/886] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:07.114652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.221334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c50bd4-b6c8-4518-8b8a-baec544251f5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.255322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.256299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.256827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.257336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.258258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:07.258786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.260345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.273775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 448/887] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:07.279942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.302754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:07.383983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68faf-7f4a-444d-8a07-516c69e48d97 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.420296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.421429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.421872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.422304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.430038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.431638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:07.447400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:07.449390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:07.450486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:07.452573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 440/888] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:07.458345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.495166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:07.565960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb855-c83c-41fc-93ee-ed4fb90c19c6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.601731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.602405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.602689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.602961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.603803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:07.604110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.605779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3b440-6977-4511-a6ea-a2d1c10434ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.624562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 449/889] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:07.630004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.643660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5799f1d-5449-4b15-bdb9-d4ed57bd5b67 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.643961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5799f1d-5449-4b15-bdb9-d4ed57bd5b67 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.644225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5799f1d-5449-4b15-bdb9-d4ed57bd5b67 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.645944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 450/890] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:07.650813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.651103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.652045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.670161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:07.671854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:07.720813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:07.742825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed81e7-09be-4c5a-86c6-aca196be28b8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.749145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 451/891] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:07.768127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.780310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.781418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.781903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.782456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.794885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:07.858860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.860675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b953b-241f-4e91-8c7d-2ba76ba9a649 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.875600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323b3de-0994-4ed1-b122-10bdb5f055dd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:07.913325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:07.914139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.914529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.914814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.915469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:07.917520 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-456c86e3-84b8-4785-af5f-8a55a5c33b99 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:07.930846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:07.931870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:07.932798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:07.933575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 452/892] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:07.938414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d608eb-5f04-4b6f-880c-5a07da2aaa21 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.938702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d608eb-5f04-4b6f-880c-5a07da2aaa21 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.938978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d608eb-5f04-4b6f-880c-5a07da2aaa21 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.943909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 453/893] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:07.943909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d608eb-5f04-4b6f-880c-5a07da2aaa21 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:07.950435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:07.971786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 441/894] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:07.973064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:07.973373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:07.973639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:07.990934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:07.992117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:08.042017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.055532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d608eb-5f04-4b6f-880c-5a07da2aaa21 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.067167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 442/895] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:08.078005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.097170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.097833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.098113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.098374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.099083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:08.099382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.100732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.106717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 454/896] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:08.107325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.195663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ce08-a2d0-4e6f-8e43-67d397944443 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.206249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.227116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eced3c-e1fa-421b-9469-b6085fb4fb0e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.231459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.232250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.232533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.232806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.246126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.247709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:08.265751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:08.266682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:08.267603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:08.267603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:08.267603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:08.267603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:08.268315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.268315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.268315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.268315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.268686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.269279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.271586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 443/897] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:08.272552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.343132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 455/898] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:08 2022] POST /identity/v3/services => generated 241 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:08.346135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.369189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.388410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66662a-77a0-4bca-ae69-5569e1ed7fca None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.445480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.446397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.446826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.447417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.449491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:08.449806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.451215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.465393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 444/899] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:08.468043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.495069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.495984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a41d6-4339-4e1b-80f1-505454547689 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.534669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.535341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.535733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.536008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.536734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:08.537081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.538572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335df5b-4f89-4e6e-88bd-8618f654f0a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.551020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 456/900] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:08.552379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e6e7-94f7-4fc4-9c4e-16cf0fae20a6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.552671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e6e7-94f7-4fc4-9c4e-16cf0fae20a6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.552941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e6e7-94f7-4fc4-9c4e-16cf0fae20a6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.555286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 457/901] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:08.555952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27e6e7-94f7-4fc4-9c4e-16cf0fae20a6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.587470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08a103-f4a3-4942-91e9-20947f0c3ed0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.624692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.625490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.625772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.626050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.626831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] RBAC: Adding query filter params (interface=public, service_id=998acb9ea40a4fa09188173f90059f82, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:08.627134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.628494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a2bc-e739-45d8-bba6-75faf6ea3631 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.630082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.642586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 445/902] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/endpoints?service_id=998acb9ea40a4fa09188173f90059f82&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:08.642586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.642586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.642586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.659922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:08.661312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:08.670391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27e6e7-94f7-4fc4-9c4e-16cf0fae20a6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.706416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.707102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.707379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.707654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.715447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.717177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:08.732126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:08.733395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:08.734327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:08.738088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 458/903] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:08.738908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.741424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 446/904] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:08.742813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.767351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.844862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02957621-fa48-4d5d-8e99-8c30e46aa4b3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.860391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bba82-6fce-4769-ad2e-87219d07d9a3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:08.877689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:08.882830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.883487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.883772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.884039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.884723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.886818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.892906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 459/905] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:31:08.896226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:08.902429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:08.903376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:08.904442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:08.904914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:08.905931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:08.906395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:08.908814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:08.929916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 447/906] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:08.935707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:09.020132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2ae4-8936-4588-80b4-c2566624ef12 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.056635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:09.061837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.062995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.064112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.064112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.064303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.066011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:09.079676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:09.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:09.081802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 460/907] 10.222.0.47 () {56 vars in 1132 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:09.086942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd45b-1eba-41d0-9fd7-5a977d2d39fb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.087242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:09.137297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.137945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.138220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.138493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.139324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] RBAC: Adding query filter params (role.id=bb0c5d919d5c41899857e7f333cda63e, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d1051f87dc384cfe86485b5e290982e2) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:09.139632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.141060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.147200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 448/908] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:08 2022] GET /identity/v3/role_assignments?user.id=d1051f87dc384cfe86485b5e290982e2&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=bb0c5d919d5c41899857e7f333cda63e => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:09.168756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:09.221854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9d571-87a2-4fd5-a8f8-51bbb6817e7b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.259851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.261294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.261403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.261677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.262403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:09.262707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.264098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.270101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 461/909] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:09.276432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:09.389948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111dba-a406-40a1-8a36-521ec68146d7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.425936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.426900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.427362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.427835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.428748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:09.429256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.430792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.436935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 449/910] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:09.442988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:09.549686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80673b-70bd-4018-961b-0926ef450b39 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.577773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:09.584877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.585569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.585851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.586128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.586919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] RBAC: Adding query filter params (interface=public, service_id=90bd614752c041058b75ea517833c9a2, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:09.587229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.589266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef29873-9df8-4de3-a401-16d03d7f8973 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.595255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 462/911] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:31:09 2022] GET /identity/v3/endpoints?service_id=90bd614752c041058b75ea517833c9a2&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:09.600945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:09.604987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:09.709417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e863b8-5601-472d-b8a1-e774123c5ccd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:09.744241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:09.744950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:09.745738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:09.746013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:09.746693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:09.748045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435a063-eeef-483a-b376-c29728a944bb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:09.754370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 450/912] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:09 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:31:10.449315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f487-f16d-4273-9078-4baf82684700 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.449823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f487-f16d-4273-9078-4baf82684700 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.449902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f487-f16d-4273-9078-4baf82684700 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:10.451534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 463/913] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:10.457249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.457541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.457821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:10.475209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:10.476396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:10.546699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:10.552642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 451/914] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:10.574717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f487-f16d-4273-9078-4baf82684700 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:10.685285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f487-f16d-4273-9078-4baf82684700 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:10.727854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:10.728639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.728921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.729344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:10.730127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:10.731508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:10.744294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:10.746019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:10.747086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 464/915] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:10.752767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:10.854608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec94f12-4b70-4c39-a09e-fabdda696fcc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:10.891320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:10.891994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.892272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.892540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:10.893288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:10.893700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:10.895086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4fddb-72a1-4c3d-8c21-813df923d68e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:10.901630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 452/916] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:10.907828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:10.975294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523a5-89ca-4ca0-98ca-6a8996d79ee0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.975652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523a5-89ca-4ca0-98ca-6a8996d79ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.975859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e523a5-89ca-4ca0-98ca-6a8996d79ee0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:10.977530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:10.977693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 453/917] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:10.984225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:10.984452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:10.984711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.001555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:11.002745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:11.021315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8eec5-395c-4faa-bf65-575fd44c40d4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.058476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.059197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.059488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.059768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.060474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:11.060780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.062175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.071306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.072420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 465/918] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:11.074170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.089530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 454/919] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:11.111104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.202278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d78-a41c-4c91-a5eb-1e2adadc59a7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.225950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.231860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf29b17-df63-418f-998f-8b537e4daccd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.250220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.251585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.252158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.252662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.254308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] RBAC: Adding query filter params (interface=public, service_id=0f0cd0704bf54adabc20c5e39ed49d55, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:11.254918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.257014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff4fd-20cb-4dd8-a827-6cb2b5922149 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.264015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 466/920] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/endpoints?service_id=0f0cd0704bf54adabc20c5e39ed49d55&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:11.265300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740bd14-8737-4a8c-b487-634b3db86f60 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.265588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740bd14-8737-4a8c-b487-634b3db86f60 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.265862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740bd14-8737-4a8c-b487-634b3db86f60 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.270897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 467/921] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:11.270897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740bd14-8737-4a8c-b487-634b3db86f60 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.274351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.275075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.275352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.275623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.276865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.278507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1d19dfcb-cac6-43fe-a7d7-4ad6efed9e0d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:11.291176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:11.292223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 12:31:11.293108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:11.297641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 455/922] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:11.298949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.299236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.299512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.316546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:11.317756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:11.375422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.378631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740bd14-8737-4a8c-b487-634b3db86f60 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.395304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 456/923] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:11.396456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.415090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.416062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.416589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.417291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.418305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.419948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.427053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 468/924] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:31:11.427210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.482933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.536844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c28ab-fbbf-4534-86ea-d52c3fd681e5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.568064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101261f-4a1b-4d3d-acc7-da15a7154930 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.581678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.582311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.582591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.582990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.583723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:11.584092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.585667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.593304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 457/925] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:11.599624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.615012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.616818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.617059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.617642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.635232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:11.643489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:11.648462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:11.649433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 469/926] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:11.651651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.667377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.741658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14badd-ca13-4d62-b522-c25572e5762a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.763190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c11f1-6a2a-41f3-a02e-ea27843aca4f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.766801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.789592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.790239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.790538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.790815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.792089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:11.792403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.794228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.805083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 458/927] 10.222.0.47 () {56 vars in 1114 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:11.807665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.809387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.810117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.810398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.810667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.811495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:11.811707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.813533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.827228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 470/928] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:11.828249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:11.921370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc05b-f0c2-4302-888d-21f272d8f23b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.934654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3676a4-2517-4445-8e4e-aeb70846ffe2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:11.936799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:11.957959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.962042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.962042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.962042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.962042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.962042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efd032-4e39-4869-b666-1bc8df2f8b6f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:11.974593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:11.975382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:11.975660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:11.975928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:11.986725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:11.988159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2e3fe404-ee40-4c28-bf16-0df3f5349d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:12.004784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:12.005890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:12.006847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:12.006847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:12.006847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:12.006847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:12.006847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 471/929] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:12.007401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185076-b12e-497c-800c-936790747863 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.007681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185076-b12e-497c-800c-936790747863 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.007954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185076-b12e-497c-800c-936790747863 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.009350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 472/930] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:12.014086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185076-b12e-497c-800c-936790747863 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.033941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.035466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 459/931] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:31:12.037194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.037477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.037829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.062930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:12.064150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:12.158022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 460/932] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:12.164103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185076-b12e-497c-800c-936790747863 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.194776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.208750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.209974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.210425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.210698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.211483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:12.211729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.214247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.226934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 473/933] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:12.233570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.311813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.318294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93553bb7-34b5-498e-b268-705ff5138838 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.348984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.358794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.359845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.360125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.360398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.361108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.362428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.368537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3efbc-d4c1-420b-a35e-c3c71821ff33 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:12.375407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:12.376394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:12.383036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 461/934] 10.222.0.47 () {56 vars in 1132 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:12.387585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.414008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.414647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.414931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.415206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.423846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.425163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:12.441322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:12.442344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:12.443907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 474/935] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:12.448461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.529884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118e62f-91f6-4145-bc4d-5cbb618fe026 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.559183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239f7bd-4c95-4b85-aac3-8bf8743e0fae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.578196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.579643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.580139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.580630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.581576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:12.582092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.584381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.593654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 462/936] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:12.599714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.605257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.605257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.605527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.605793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.606646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:12.606955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.608734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.613423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.635877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 475/937] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:12.637388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.641398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.715721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae42f7e-2e01-44f1-8d11-70463c962991 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.752646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c7322-069b-4123-a358-0716676163f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.753423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c7322-069b-4123-a358-0716676163f1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.753837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c7322-069b-4123-a358-0716676163f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.754112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c7322-069b-4123-a358-0716676163f1 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.754921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c7322-069b-4123-a358-0716676163f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:12.755313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c7322-069b-4123-a358-0716676163f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.756736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c7322-069b-4123-a358-0716676163f1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.766659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4252a-fff5-4963-8dad-9d6357b96923 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.767092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 463/938] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:12.768488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c7322-069b-4123-a358-0716676163f1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.810208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.811857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.812343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.812814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.826542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.828736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:12.850760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:12.851695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:12.852604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:12.852604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:12.852604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:12.852604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:12.861245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 476/939] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:12.862034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:12.873113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.886846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c7322-069b-4123-a358-0716676163f1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:12.920549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:12.924243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:12.924981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:12.925379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:12.925726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:12.926544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:12.928036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:12.977825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c83fe-ed30-4723-8ff6-6409bae9f6da None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:13.009339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 464/940] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:13.030769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:13.031440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.031720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.031989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.033016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:13.033352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:13.034879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:13.049318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 477/941] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:13.055271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:13.188117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243908d1-d3ab-4683-a506-0f344b0cfb5c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:13.227461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:13.228329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.228613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.228957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.236568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:13.238053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:13.252708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:13.253825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:13.254746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 465/942] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:13.259975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:13.367864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:13.372227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fbd42-b43a-495e-aea9-d6ea46b11315 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:13.407341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:13.408003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.408285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.408562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.409433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:13.409757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:13.411188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693b312-863d-4414-b81e-f3e02cad6455 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:13.429152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 478/943] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:13.430917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:13.440863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:13.534242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6d805-60d9-4038-82c4-8a8391f83552 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:13.569824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:13.571086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.571564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.572032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d1051f87dc384cfe86485b5e290982e2/roles/bb0c5d919d5c41899857e7f333cda63e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.574624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=d1051f87dc384cfe86485b5e290982e2, role_id=bb0c5d919d5c41899857e7f333cda63e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:13.576285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:13.596004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f93a-5912-4a09-ab90-c100e96fd53b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.596495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f93a-5912-4a09-ab90-c100e96fd53b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.596572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f93a-5912-4a09-ab90-c100e96fd53b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.598135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 479/944] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:13.602741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.603032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.603330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.621749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:13.622940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:13.639809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 466/945] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:31:13 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/d1051f87dc384cfe86485b5e290982e2/roles/bb0c5d919d5c41899857e7f333cda63e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:31:13.706413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 480/946] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:13.735630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:13.809757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:13.852074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53905f9-365f-4f8b-a2b2-bf1abf97fcbc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:13.889150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:13.890012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:13.890295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:13.890561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:13.891218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:13.892570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:13.905695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:13.906714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:13.907989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 467/947] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:13.913362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.003813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:14.013954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e9ee-0e20-42d2-8bd4-641caf4d657d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.049196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.049860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.050139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.050465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.051160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:14.051510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.052867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2951f02-c8ef-4156-97de-c81955785d91 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.058750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 481/948] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:14.064344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.171567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde690a-edee-4635-a2bd-f704e54ee790 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.200573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:14.209271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.210203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.210642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.211055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.211957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:14.212416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.215475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bde8f-95d2-4a49-85d7-f71f9e1e5c4e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.215752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bde8f-95d2-4a49-85d7-f71f9e1e5c4e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.216272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bde8f-95d2-4a49-85d7-f71f9e1e5c4e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.217824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 482/949] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:14.224842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.226422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.226917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.227024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 468/950] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:14.227488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.232855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.248560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:14.250590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:14.353246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 483/951] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:14.362363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd49ac-0412-41a3-901d-55ccda36a9f3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.377823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.415280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.415935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.416213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.416531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.417287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.418638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.471765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:14.487014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaa15a-b349-4d55-88a5-775b8756014d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.516510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 469/952] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:31:14.541744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.542955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.543331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.543612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.544346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.545845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.547364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING keystone.server.flask.common [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 16 12:31:14.598635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.sql.core [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 12:31:14.598635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 16 12:31:14.598635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [parameters: {'id': 'ca38598af5374a5ba4005da44c91e4a3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 16 12:31:14.598635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80440) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca38598af5374a5ba4005da44c91e4a3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca38598af5374a5ba4005da44c91e4a3. Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:31:14.616407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 16 12:31:14.617357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application yield resource Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise self.value Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application session.commit() Jul 16 12:31:14.624101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.flush() Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application compat.raise_( Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 12:31:14.626225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 12:31:14.628359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:31:14.635921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [parameters: {'id': 'ca38598af5374a5ba4005da44c91e4a3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:14.643964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:14.646071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca38598af5374a5ba4005da44c91e4a3. Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:14.648858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 484/953] 10.222.0.47 () {60 vars in 1140 bytes} [Sat Jul 16 12:31:14 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:14.660390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.705524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:14.759099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06807e9-d3cf-46d0-a568-b2c31f4a7c51 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.796491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.797591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.798026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.798445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.805731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.808126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-95803f5c-b942-47ba-91c9-05588e6e1d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:14.822942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:14.824072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:14.825351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:14.827493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 470/954] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:14.833769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:14.952327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39af4-6cfd-4f1d-8f4b-e7f8234231aa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:14.988719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:14.989412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:14.989690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:14.989962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:14.990717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:14.991021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:14.992648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53454-f938-47ab-975d-f256416914fb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:14.997127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:15.014258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 485/955] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:15.174729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3bb58-ec25-45c5-9999-6d9f0085dfa9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.175230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3bb58-ec25-45c5-9999-6d9f0085dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.175306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3bb58-ec25-45c5-9999-6d9f0085dfa9 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.176627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 471/956] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:15.183019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.183297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.183643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.203853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:15.205616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:15.279124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 486/957] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:15.301027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3bb58-ec25-45c5-9999-6d9f0085dfa9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:15.410418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3bb58-ec25-45c5-9999-6d9f0085dfa9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:15.420943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:15.446924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:15.448336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.448620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.448888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.449572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:15.450925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:15.463041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:15.464705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:15.465604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 472/958] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:15.473528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:15.579214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc008e7b-990e-4e1c-af10-a25953891446 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:15.610970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:15.624030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:15.625003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.625568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.632837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.632837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:15.632837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:15.632837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325d8cd-e357-4cca-9b1b-43fa81f8dec4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:15.634415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 487/959] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:15.641123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:15.751671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f23739-ccd2-40e5-a23e-781efcf388a4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:15.788844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:15.789538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.789820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.790088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.790789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:15.791094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:15.793672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971e7e0-9e38-466c-b0c5-8babc643821c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:15.798868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 473/960] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:15.807606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b0ae-5497-4271-8320-53ba04910436 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.808165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b0ae-5497-4271-8320-53ba04910436 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.808484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b0ae-5497-4271-8320-53ba04910436 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.810300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 488/961] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:15.816498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:15.816830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:15.817082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:15.836209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:15.837469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:15.873274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:15.914810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 474/962] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:15.939586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b0ae-5497-4271-8320-53ba04910436 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.051193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b0ae-5497-4271-8320-53ba04910436 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.085882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.086688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.086965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.087237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.099083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:16.107463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:16.108440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:16.109437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 489/963] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:16.115040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.220092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0f978-96de-4e51-8d6b-bcff908eac42 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.257693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.258378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.258655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.258926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.259685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:16.260029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.262295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.271518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 475/964] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:16.277456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.331482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:16.385164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760ee8c-1d64-4030-8430-a4d759a59c3e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.420130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.421278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.421760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.422251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.433990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.435702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:16.452865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:16.453808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:16.454715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:16.454715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:16.454715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:16.454715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:16.456892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 490/965] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:16.469148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.498820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:16.573538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953cb3a-5355-41c9-b946-e1e6efc42a5d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.610777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.611446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.611780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.612054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.612782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:16.613106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.614476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311e291-015d-466a-b829-a48aa872ceb6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.627197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 476/966] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:16.632757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.686653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35766-f413-4dc2-86cd-eb527aed0e8b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.687026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35766-f413-4dc2-86cd-eb527aed0e8b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.687219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35766-f413-4dc2-86cd-eb527aed0e8b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.688692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 477/967] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:16.694325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.694680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.694874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.711740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:16.712926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:16.743881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093141c-b735-455e-9c34-c124fad9b8e4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.746604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:16.780112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.781529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.782029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.782470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.789132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 478/968] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:16.796917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.798491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.809224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:16.815589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:16.816533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 12:31:16.817555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:16.820118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 491/969] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:16.826699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:16.828468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:16.919631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac28d-0641-42e1-a7c8-900e7db13b8e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.934454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ab375-2836-4399-a200-58b0217ff3d4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:16.956941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.957618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.958001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.958429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.959176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.960744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.972780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:16.973673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:16.974126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:16.974546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:16.974864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:16.975950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:16.976819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 479/970] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:16.983395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:16.983395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:16.983395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:16.983750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.008526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 492/971] 10.222.0.47 () {56 vars in 1149 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:17.009755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.035453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.091099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.112752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc8c6-2ef3-4eab-bee5-eb6a8e125b1b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.118589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512fdbd-7377-47ba-9165-308ac87cdc52 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.151237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.152215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.152525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.152865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.153698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:17.154070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.155536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.156695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.157562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.158116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.158566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.165320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 480/972] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:17.166886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.253367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.255570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47a8d6-c683-492f-a495-3965ca435fc9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.279865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a0b6-994a-4cc3-94db-c9c3ff5bcfda None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.289793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.316643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.317339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.317626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.317900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.318609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:17.318916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.320272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d28d7-bf3f-4c81-bdc4-e1c78e13bcc0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.326074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 481/973] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:17.327677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e95e6-f062-4b19-8a7a-854b458714fa None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.328184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e95e6-f062-4b19-8a7a-854b458714fa None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.328659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e95e6-f062-4b19-8a7a-854b458714fa None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.330384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 482/974] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:17.332574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e95e6-f062-4b19-8a7a-854b458714fa None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.349908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.393620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 493/975] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:17.393620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb655-deca-490f-b873-729da6f249e8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.393620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb655-deca-490f-b873-729da6f249e8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.393620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efb655-deca-490f-b873-729da6f249e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.413234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-42efb655-deca-490f-b873-729da6f249e8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:17.414687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-42efb655-deca-490f-b873-729da6f249e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:17.471322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e95e6-f062-4b19-8a7a-854b458714fa None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.498703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 494/976] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:17.499893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efb655-deca-490f-b873-729da6f249e8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.531857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.533185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.533472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.533891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.534831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] RBAC: Adding query filter params (interface=admin, service_id=0f0cd0704bf54adabc20c5e39ed49d55, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:17.535146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.536728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.545543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 483/977] 10.222.0.47 () {56 vars in 1249 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/endpoints?service_id=0f0cd0704bf54adabc20c5e39ed49d55&interface=admin®ion_id=RegionOne => generated 193 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:17.546236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.569099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.642998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efb655-deca-490f-b873-729da6f249e8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.644689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.679102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807002bc-8b30-4295-a65d-a52c1cca5ec8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.688915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.690274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.690559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.690835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.704800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.707006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.717423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.718687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.719386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.719836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.720706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.722418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.723914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING keystone.server.flask.common [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:17.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:17.727802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:17.728792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:17.728792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:17.728792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:17.728792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:17.732900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 495/978] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:17.733780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.769121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.sql.core [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 12:31:17.769121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 16 12:31:17.769121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [parameters: {'user_id': 'e6f72f00d2c54c04800f849e3ad3463a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 16 12:31:17.769121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 16 12:31:17.777111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:31:17.778066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application yield resource Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise self.value Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application session.commit() Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 12:31:17.779080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.flush() Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application compat.raise_( Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 12:31:17.780010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 12:31:17.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 12:31:17.781855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e6f72f00d2c54c04800f849e3ad3463a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:17.782710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 16 12:31:17.783533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 484/979] 10.222.0.47 () {60 vars in 1131 bytes} [Sat Jul 16 12:31:17 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:17.784489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.845879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55272ec4-1e77-4b8f-adea-e7989816bd4e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.847424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.881390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.882306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.882590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.882879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.883541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.885452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.887662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02e000-619b-4d08-b12f-c76239d12433 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:17.891689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 496/980] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:31:17.892483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:17.941190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:17.941876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:17.942214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:17.942497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:17.943242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:17.943609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:17.946717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:17.950530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:17.971245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 485/981] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:17.982656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:18.024580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da769a-3675-44a4-ac0e-afba3bf40231 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.067438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.068219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.069537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.069537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.078179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.079981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fb243fd5-b6be-410a-a8a2-1503758e5064 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:18.097423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:18.098623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:18.099594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 497/982] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:18.100261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768ab-d03b-468e-b960-4bdb07f52fda None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.100261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768ab-d03b-468e-b960-4bdb07f52fda None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.100397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768ab-d03b-468e-b960-4bdb07f52fda None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.101897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 498/983] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:18.105139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6624be-4706-4518-a59a-296c53672496 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.107185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d768ab-d03b-468e-b960-4bdb07f52fda None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:18.136729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:18.146393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.146849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.149627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.149627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.154945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.156403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-81ab9c55-47c5-4764-9063-82a0b5048c2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:18.171817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:18.172770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 486/984] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.173744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.174357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.195970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:18.198155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:18.224542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d768ab-d03b-468e-b960-4bdb07f52fda None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.255862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:18.260780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.261481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.261765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.262042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.262849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:18.263163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.264620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.273175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 487/985] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:18.280962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:18.314157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 499/986] 10.222.0.47 () {56 vars in 1101 bytes} [Sat Jul 16 12:31:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:18.315032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:18.407793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98379d9-2c0c-496e-9c8f-6ce840d4e22b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.429911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:18.449251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab66dd-ef37-484c-aa83-7a7159928781 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.456495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.457196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.457477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.457748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.458548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:18.458850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.461401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.482887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 488/987] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:18.485787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:18.500324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.501009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.501322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.501967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.502697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.504395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215c438-24f4-420f-b0b2-2fb5f4509ad1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.540503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:18.578902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 500/988] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:18 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:18.610977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5b00-19fa-420d-aec4-ce5d1d77c0a6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:18.711050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:18.711724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:18.712004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:18.712300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:18.713226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] RBAC: Adding query filter params (role.id=bb0c5d919d5c41899857e7f333cda63e, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=d1051f87dc384cfe86485b5e290982e2) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:18.713546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:18.715003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8e502-15b2-451d-b0de-d8324d667f78 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:18.729097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 489/989] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:18 2022] GET /identity/v3/role_assignments?user.id=d1051f87dc384cfe86485b5e290982e2&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=bb0c5d919d5c41899857e7f333cda63e => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:19.958650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9aac8-818f-473b-a1d8-dbf02df3d559 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:19.959176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9aac8-818f-473b-a1d8-dbf02df3d559 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:19.959256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9aac8-818f-473b-a1d8-dbf02df3d559 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:19.960740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 501/990] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:19.966716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:19.967110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:19.967272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:19.983632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:19.984808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:20.052553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 490/991] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:20.075174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9aac8-818f-473b-a1d8-dbf02df3d559 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.186342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9aac8-818f-473b-a1d8-dbf02df3d559 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:20.221900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:20.222625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.222906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.223178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.223825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:20.225215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:20.237133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:20.238118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 12:31:20.239054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.240422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 502/992] 10.222.0.47 () {56 vars in 1126 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:20.244361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.279277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:20.345196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237dba-f063-4bf9-b1c0-d807737dd2ff None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:20.378781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:20.379441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.379721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.379992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.380698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:20.381001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:20.382380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:20.392050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 491/993] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:20.393250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.405508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:20.493376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7a1c-0f39-430c-8f1c-2e28110e8036 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:20.527079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:20.527749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.528032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.528307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.529007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:20.529342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:20.530700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a12d7-5c00-4f8d-8078-0814c7f32d98 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:20.540163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 503/994] 10.222.0.47 () {56 vars in 1122 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:20.542035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.637559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9a81-d34a-4d82-aa37-d11e913ac200 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.637897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9a81-d34a-4d82-aa37-d11e913ac200 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.638140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9a81-d34a-4d82-aa37-d11e913ac200 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.639786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 504/995] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:20.644541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149f65-7740-464a-a7e2-ff75e3107d29 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.644828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149f65-7740-464a-a7e2-ff75e3107d29 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.645141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b149f65-7740-464a-a7e2-ff75e3107d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.646236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe01b-5c5d-4503-8c96-1f67cb3f73b4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:20.648032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:20.662971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2b149f65-7740-464a-a7e2-ff75e3107d29 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:20.665211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2b149f65-7740-464a-a7e2-ff75e3107d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:20.689048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:20.690147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.690559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.690980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.691976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:20.694050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8f4f1-1ec0-44d6-be83-b20199005f84 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:20.735822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 505/996] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:20.753626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b1c32-62f9-4d80-92b8-2fa94bb9509c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.753835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b1c32-62f9-4d80-92b8-2fa94bb9509c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.754107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b1c32-62f9-4d80-92b8-2fa94bb9509c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.755462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 506/997] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:20.757866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b1c32-62f9-4d80-92b8-2fa94bb9509c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.768238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 492/998] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:20.768454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:20.768538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.768538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.768538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.786030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:20.787264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:20.872618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 493/999] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:20.881936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b1c32-62f9-4d80-92b8-2fa94bb9509c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:20.896315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.925584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:20.926382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:20.926670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:20.926943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:20.927589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:20.928949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:20.941768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:20.942785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:20.944598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 507/1000] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:20.949629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:20.996710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:21.013420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90ccbb-daff-4df1-a510-e2b56a353746 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.022696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:21.048395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.049215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.049496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.049776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] PATH_INFO: `/v3/services/image` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.050425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.051785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.059302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef918b7-f2f5-4330-8c6b-3b23387288b5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:21.063874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:21.068854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:21.070243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 494/1001] 10.222.0.47 () {56 vars in 1105 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:21.071076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:21.100857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.101543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.101822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.102097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.102797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:21.103098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.105148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.114832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 508/1002] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:21.115663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:21.189798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd7af1-d77a-4c58-9e71-ddf2b61e79ff None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.222428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d378b-b893-402a-9585-ae2c54170f53 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.226458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.227127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.227404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.227675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.228401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] RBAC: Adding query filter params (name=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:21.228816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.230977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.241386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 495/1003] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:21.242427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:21.251780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:21.258085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.258745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.259037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.259316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.262486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:21.262486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.262659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:21.262752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.267242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 509/1004] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:21.272940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:21.352456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03b7d-eb63-44e9-ac69-a43c1a2a2016 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.376817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1df8d-6e45-43c3-bf9a-efbaf1aafa0b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.386090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.386753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.387029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.387307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.388015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:21.388324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.389712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.394095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 496/1005] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:21.414045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.415073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.415580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.416577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.417602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] RBAC: Adding query filter params (interface=public, service_id=617666a85f8747b79528b296831ffc84, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:21.418067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.419681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f103-b176-41bc-b5e3-4c0ad6883fcb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.426262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 510/1006] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:31:21 2022] GET /identity/v3/endpoints?service_id=617666a85f8747b79528b296831ffc84&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:21.432454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:21.521839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:21.545634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4d59-0d90-4dba-a1b4-181ee5fce6a7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:21.583559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:21.584219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:21.584497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:21.584769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:21.585479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:21.586822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106d5c0-b911-4899-b91c-b04a6b6eba60 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:21.594004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 497/1007] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:21 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:31:22.810158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e655db-a6c0-4217-b712-057334dde938 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:22.810711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e655db-a6c0-4217-b712-057334dde938 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:22.810711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e655db-a6c0-4217-b712-057334dde938 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:22.812085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 511/1008] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:22.817168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:22.817582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:22.817741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:22.835244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:22.836419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:22.905572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 498/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:22.928243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e655db-a6c0-4217-b712-057334dde938 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:22.989903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:23.032990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e655db-a6c0-4217-b712-057334dde938 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:23.066262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:23.066971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.067249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.067517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.068673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:23.070120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af41d96-b1a3-49f9-bec5-85fdf3e8a8dd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:23.071934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 512/1010] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:23.078261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:23.149081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:23.188935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f999a1-651a-41c7-80ba-9179c7878c8f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:23.225431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:23.226084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.226363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.226635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.227345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:23.229070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2a140-e780-4b5c-9f3b-e0d88844abc4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:23.317155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 499/1011] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:23 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:23.435162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8bde-311e-4b47-9e8f-25f266508f62 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.435514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8bde-311e-4b47-9e8f-25f266508f62 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.435728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8bde-311e-4b47-9e8f-25f266508f62 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.437341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 513/1012] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:23.442923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cca9-0e89-4b01-8661-9384f85c86f0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.443216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cca9-0e89-4b01-8661-9384f85c86f0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.443501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cca9-0e89-4b01-8661-9384f85c86f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.462213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2b49cca9-0e89-4b01-8661-9384f85c86f0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:23.464085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2b49cca9-0e89-4b01-8661-9384f85c86f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:23.537118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 500/1013] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:23.557538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8bde-311e-4b47-9e8f-25f266508f62 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:23.614806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7dab4-6e3a-4065-95ee-40023955ace2 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.615360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7dab4-6e3a-4065-95ee-40023955ace2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.615631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7dab4-6e3a-4065-95ee-40023955ace2 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.617671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 501/1014] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:23.622833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.623113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.623384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.640221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:23.641444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:23.673645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8bde-311e-4b47-9e8f-25f266508f62 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:23.693774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:23.700565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:23.709431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:23.710328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.710794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.711244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.712142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:23.714334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 502/1015] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:23.714701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:23.738046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:23.758163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 514/1016] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:23 2022] POST /identity/v3/services => generated 242 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:23.850934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af1037-4b8a-44f9-a4d9-3a3f5a78dae2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:23.895744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:23.896456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:23.896731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:23.896998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:23.897685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:23.899049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:23.912697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:23.914120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:23.916133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 503/1017] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:23.920883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:24.031097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fa3b5-493b-4720-b8de-67c299acacf0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:24.065116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:24.065949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:24.066339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:24.066721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:24.067533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:24.067946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:24.069434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:24.075508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 515/1018] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:24.081308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:24.128376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:24.189348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd496b-9dca-4691-bbbf-99a7ad89d7e8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:24.224702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:24.225668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:24.226098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:24.226520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:24.227366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:24.227817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:24.229383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbde31d-de62-4cd7-a858-8ee8474db9fc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:24.234408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 504/1019] 10.222.0.47 () {56 vars in 1112 bytes} [Sat Jul 16 12:31:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:24.240003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:24.290538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:24.341406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f173c-0977-4be8-9999-809ec2e01dd7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:24.375833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:24.376481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:24.376760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:24.377059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:24.377792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:24.379134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f10a79-d845-4cf0-a68d-8f1bc7a4634a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:24.454990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 516/1020] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:25.391123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8dc3-384c-4625-9127-6d48d29bd1d6 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.391654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8dc3-384c-4625-9127-6d48d29bd1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.391654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8dc3-384c-4625-9127-6d48d29bd1d6 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.393214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 505/1021] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:25.398197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.398590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.398777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.416893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:25.418116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:25.490562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 517/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:25.514995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e8dc3-384c-4625-9127-6d48d29bd1d6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:25.618333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e8dc3-384c-4625-9127-6d48d29bd1d6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:25.628054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:25.654897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:25.655731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.656014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.656289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.657632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:25.659218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65757964-bbc8-47f3-9842-6ef5ff23e93f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:25.661166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 506/1023] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:25.666991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:25.698589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:25.769356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3997ad-1ea0-42a4-897c-13a941c2a7e7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:25.802793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:25.803196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.803481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.803756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.804302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3685e7-7974-40ea-9b0c-b02e7ffc3fa8 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.804462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:25.804622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3685e7-7974-40ea-9b0c-b02e7ffc3fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.804862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3685e7-7974-40ea-9b0c-b02e7ffc3fa8 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.805972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:25.809151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 507/1024] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:25.812148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:25.812433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:25.812708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:25.831180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:25.832348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:25.897885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 508/1025] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:25.918554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 518/1026] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:31:25 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:25.921020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.033114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ad390-0343-4662-89ed-195d7ce89e4a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.042034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:26.072426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.073975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.074254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.074526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] PATH_INFO: `/v3/services/image` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.075175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:26.076527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:26.088291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:26.089351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:26.090227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:26.094806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 509/1027] 10.222.0.47 () {56 vars in 1105 bytes} [Sat Jul 16 12:31:25 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:26.095935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.196201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:26.208353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad1678-eeb4-49be-9cba-8ce56a411efb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.243335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.244551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.244831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.245132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.245840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] RBAC: Adding query filter params (name=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:26.246202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:26.247767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:26.252865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 519/1028] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:26.258743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.364069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a447-ae9d-48de-a8e7-cc55f78c3304 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.398530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.400598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.401201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.401769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.402694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] RBAC: Adding query filter params (type=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:26.403216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:26.404792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:26.415069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 510/1029] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:26.416851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.519927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad9b6b-26cc-45b7-9059-bf4deeb29e63 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.554353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.555064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.555351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.555628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.556486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] RBAC: Adding query filter params (interface=public, service_id=e1d8d243b42f43bc8187dfb04646a064, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:26.556796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:26.558195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dda56-ab22-4de7-83dd-40a8765c5edd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:26.567804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 520/1030] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/endpoints?service_id=e1d8d243b42f43bc8187dfb04646a064&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:26.569438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.575818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a155-81e6-49af-8234-12bcf9a6d959 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.576110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a155-81e6-49af-8234-12bcf9a6d959 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.576389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a155-81e6-49af-8234-12bcf9a6d959 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.577881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 521/1031] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:26.582824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.583326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.583825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.601497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:26.602918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:26.608219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:26.615327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:26.677274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 522/1032] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:26.678246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2052545-591c-499e-a8f8-768c99e34879 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.700158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.715838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.716642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.716771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.717069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.717843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:26.719136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:26.725843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 511/1033] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:31:26.816281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9f33-ccb0-4cb8-882f-82e979715c17 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:26.853939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:26.854881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:26.855149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:26.855416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:26.866838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:26.876771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:26.877796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:26.878650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 523/1034] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:26.884002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:26.985490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:26.989701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91056d87-804b-4995-80f2-7d18d88cb0d1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.023962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.025540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.026368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.026856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.027838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:27.028355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.029958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:27.038746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 512/1035] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:27.045173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:27.054683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:27.147877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898fa4-4985-4ce5-ac38-f3ced6c47e77 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.191058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.191854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.192137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.192409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.203603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.206794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:27.224566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:27.225685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:27.226615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:27.226615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:27.226615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:27.226615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:27.226615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 524/1036] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:27.233210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:27.334028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabee3-7690-43dc-be98-7d31901a2914 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.367514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.368168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.368444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.368791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.369528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:27.369827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.371168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:27.382959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 513/1037] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:27.388354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:27.447475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:27.490795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424810-6698-44b0-b015-8c54f093dbf7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.523759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.524545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.525488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.525763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.533186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.534600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:27.548188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:27.549197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:27.550189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:27.551508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 525/1038] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:31:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:27.555062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:27.577887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:27.659392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb19f10-875f-41f1-b492-b4f14a469462 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.695956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.696645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.697014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.697435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.698330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:27.698589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.700012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a292f3-d1f1-4a5e-bacc-4ff7758c5623 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:27.722753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 514/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:31:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:27.723701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:27.826676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301fde4-7ffe-46a6-93ca-c1224e3333ae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:27.845156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:27.863553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:27.864932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:27.864932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:27.864932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:27.934711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:27.936160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.018244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99c56-90e0-4dc9-a0b9-e0267d7bdc6b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.018456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99c56-90e0-4dc9-a0b9-e0267d7bdc6b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.018730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99c56-90e0-4dc9-a0b9-e0267d7bdc6b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.020091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 515/1040] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:28.024843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.025162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.025440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.036209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 526/1041] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 12:31:28.041751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.046295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:28.047474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:28.060104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.118539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 516/1042] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:28.140762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.151905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.152414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf231db-6cba-44ab-95dd-c19ca0646d68 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.192916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.193934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.194063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.194258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.205424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.206849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:28.225537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:28.226484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:28.227510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:28.227510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:28.227510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:28.227510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:28.227510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 527/1043] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:28.232396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.252836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21112d81-778a-4bc0-a672-ab92c8752cd9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.292959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.293712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.293994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.294265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.308525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:28.308983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:28.316108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:28.317092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:28.317960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 517/1044] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:28.323665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.349014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b37818-aa31-4be9-8341-0cd0c056d36d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.388094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.388749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.389079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.389336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.390046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:28.390352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.391698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.396656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.403653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 528/1045] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:28.409606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.436364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5a720-c5f0-4c94-929b-781e6fddd270 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.472545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.473368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.473650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.473930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.474678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:28.474989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.476348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.490345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 518/1046] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:28.491452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.518523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0ee8-b197-4d78-8ec9-9eda0023b849 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.548714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.555034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.562117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.562117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.562117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.564372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.566063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:28.582209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:28.583323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:28.584388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:28.592203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 529/1047] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:28.593074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.598196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e05e-22f7-40c6-a274-9439efb14e61 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.627919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.635339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.636273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.636610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.636888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] PATH_INFO: `/v3/users/3d1c5878a0cd443db3ea7e8813f421b5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.650985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d1c5878a0cd443db3ea7e8813f421b5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.652449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.654406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 519/1048] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/users/3d1c5878a0cd443db3ea7e8813f421b5 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:28.659856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.695459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc769bc-6676-4d93-b1be-cdc02692b998 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.730483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.731137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.731416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.731687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.732465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:28.732769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.734208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.746320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 530/1049] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:28.752576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:28.770160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c72fa-9ec7-47ad-a8ca-8e61920022e4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.801321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.804097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.804744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.805021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.805317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.851802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55477d3-f3e2-4be6-9b25-849e43a6bac9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:28.860226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:28.872983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.874450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.885442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:28.886096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.886371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.886642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.887486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] RBAC: Adding query filter params (role.id=bb0c5d919d5c41899857e7f333cda63e, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:28.887795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:28.889236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0ab5-4adf-4fea-b9bc-f505640c8ac8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:28.895077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 531/1050] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=bb0c5d919d5c41899857e7f333cda63e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:28.896307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe9b-090e-451c-97de-8f3e2846d4e6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.896594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe9b-090e-451c-97de-8f3e2846d4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.896869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe9b-090e-451c-97de-8f3e2846d4e6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.898312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 532/1051] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:28.903868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:28.904169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:28.904443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:28.926555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:28.927791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:28.989788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 520/1052] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 12:31:28.998067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.017063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 533/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:29.032517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.082107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:29.110222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:29.124817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00086df5-47f0-465a-aeb8-127171d3d2a8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.137943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d846-30d8-4ba6-92cf-f36b48c7e510 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.160478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.161369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.161652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.161927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.163119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.164538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.172054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 521/1054] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:28 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:29.173261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.188714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.189552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.189838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.190114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] PATH_INFO: `/v3/services/image` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.190769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.192134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:29.204091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:29.205128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 12:31:29.206174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:29.210344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 534/1055] 10.222.0.47 () {56 vars in 1105 bytes} [Sat Jul 16 12:31:29 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:29.211361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.286930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f8d10-4306-4e73-bda7-78a30bd5b8ec None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.312402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819062b-b8c5-4749-ae81-2efc09405031 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.321556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:29.326948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=a28ab568f5584dcd98e1232dfd0dc445, user.id=3d1c5878a0cd443db3ea7e8813f421b5) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.334901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.344231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 522/1056] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:29 2022] GET /identity/v3/role_assignments?user.id=3d1c5878a0cd443db3ea7e8813f421b5&scope.project.id=a28ab568f5584dcd98e1232dfd0dc445&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:29.356048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.356747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.357031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.357336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.358070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:29.358379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.359796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.370985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 535/1057] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:29.372820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.406482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:29.491201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dee00-ee48-4580-b5ae-6314c0e37e8d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.531307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.531977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.532258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.532529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.533260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] RBAC: Adding query filter params (type=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:29.533567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.534925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f225-e47b-45ee-ad83-0ff18d162c05 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.544536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 523/1058] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:29.546147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:29.649070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb018-f27e-423d-bf92-eec2ea61a6f1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:29.684838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:29.685527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:29.685809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:29.686082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:29.686830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:29.688188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb2aba-47e4-41fa-8530-3b8edbe6a712 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:29.769647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 536/1059] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:31:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:31:30.912482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e428c5-5b70-490b-b116-371b8fbee781 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:30.913010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e428c5-5b70-490b-b116-371b8fbee781 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:30.913010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e428c5-5b70-490b-b116-371b8fbee781 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:30.914465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 524/1060] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:30.921070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:30.921523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:30.921750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:30.938238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:30.939424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:30.961759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.012721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 537/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:31.035355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e428c5-5b70-490b-b116-371b8fbee781 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.121372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.138457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e428c5-5b70-490b-b116-371b8fbee781 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.175122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.178553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.178553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.178553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.189753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:31.198344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:31.199291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:31.200178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:31.201649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 525/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:31.205551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.305400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6611bb-c598-4870-85ac-db1b37e94bc8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.340348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.341015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.341328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.341601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.342345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:31.342650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:31.344013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6bdb2-e5ac-4892-8dd6-eb9800a778d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:31.352394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 538/1063] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:31.358071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.407440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280bcf-1129-426a-8e94-e20df6ef826f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.407757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280bcf-1129-426a-8e94-e20df6ef826f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.407997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280bcf-1129-426a-8e94-e20df6ef826f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.409392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 539/1064] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:31.414315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.414600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.414876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.433303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:31.434392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:31.445154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.463755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40322ea0-74d9-4465-b096-f6af6ce76cfb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.503435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.504222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.504500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.504951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 540/1065] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:31.509242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.520344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:31.521817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:31.525222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.527344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:31.543383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:31.544476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:31.545433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:31.545433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:31.545433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:31.545433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:31.545433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 526/1066] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:31.549942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.646976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540e9e5-99e9-417b-a655-44284f5976d8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.664538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58226-37ba-4a6a-acbc-73f59afba41b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.683242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.684088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.684309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.684579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.694616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:31.701149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.701802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.702081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:31.702320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:31.703234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:31.704086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:31.705269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.705269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:31.705269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:31.705269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:31.708740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 541/1067] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:31.709592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.719251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 527/1068] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:31.721941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.726227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.787189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.818611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a30f4c-4131-4b9c-945e-e2d1877d2980 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.832415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19366-2984-45e8-8278-3efc36da1bc0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.852189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.853625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.854114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.854396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.855135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:31.855438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:31.856786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5a08e-7e22-4577-919c-bddf3a6fecfe None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:31.864699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 542/1069] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:31.865940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ba92-fe27-487b-ae6a-81f71bd2ccd7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.866226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ba92-fe27-487b-ae6a-81f71bd2ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.866495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ba92-fe27-487b-ae6a-81f71bd2ccd7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.867715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 543/1070] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:31.871182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13ba92-fe27-487b-ae6a-81f71bd2ccd7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:31.872063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:31.872964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.873282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.873560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.880976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:31.882442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2e3f0610-ab1c-4cce-8f21-507ca5ce8d43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:31.897635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:31.898674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:31.899699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 528/1071] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:31.900561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:31.900749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:31.901023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:31.917479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:31.918647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:31.940378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:31.974411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13ba92-fe27-487b-ae6a-81f71bd2ccd7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:31.985126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 529/1072] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:31.985954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.011601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.012463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.012740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.013004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] PATH_INFO: `/v3/users/3d1c5878a0cd443db3ea7e8813f421b5` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.014216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] RBAC: Authorizing `identity:get_user(user_id=3d1c5878a0cd443db3ea7e8813f421b5)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.015623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.017406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 544/1073] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/users/3d1c5878a0cd443db3ea7e8813f421b5 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:32.022004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.035884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.088613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03bdde-820c-4f69-9d66-c3321518e727 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.119579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fa514-e0b2-489b-826d-75f245c9de36 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.121608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.122992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.123276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.123547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.124366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:32.124671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.126094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.143743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 530/1074] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:31:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:32.144527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.157885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.158558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.158842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.159120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.162545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.227627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.229155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.243472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cde18b-8b14-4139-ab20-14de9f695e22 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.245916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.279540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.280395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.280677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.280958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.282177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.283590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.285327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 531/1075] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:32.286084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.329606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 545/1076] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:32.330413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.365862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.384812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c2d5-2669-4759-a73a-f0568f36ad7e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.417594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.418376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.418655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.418927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.428790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.429444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57149f-6279-4bb9-a47b-7e387a5e41dc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.430256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:32.449853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:32.450989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 532/1077] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:32.451925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.466550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.467429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.467704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.467977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445/users/3d1c5878a0cd443db3ea7e8813f421b5/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.469938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] RBAC: Authorizing `identity:create_grant(project_id=a28ab568f5584dcd98e1232dfd0dc445, user_id=3d1c5878a0cd443db3ea7e8813f421b5, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.471365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.477798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.534371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 546/1078] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:31:32 2022] PUT /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445/users/3d1c5878a0cd443db3ea7e8813f421b5/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:31:32.535217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.551131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ae8e-0984-406b-b26a-2626c88f7541 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.601740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.602557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.602822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.603927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.615899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.617405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:32.638999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:32.640041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:32.640947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:32.640947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:32.640947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:32.640947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:32.640947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 533/1079] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:32.646416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.668083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.674260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b797a-e867-4bb4-896d-e55da2c8130c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.717222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.718329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.718611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.719004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.719767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:32.720075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.721543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.734178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 547/1080] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:32.739301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:32.740890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:32.763879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df387ff6-7b93-4307-84f0-0355f6c30d50 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.802465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.803132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.803464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.803737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.804485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:32.804788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.806797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.819016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 534/1081] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:32.855967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cec0-06fa-4b5c-9c1d-c75d389a5444 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:32.893173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:32.893985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:32.894275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:32.894544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:32.902213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:32.903694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:32.920244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:32.921298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:32.927470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 548/1082] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:32.927985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:33.041419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf5680-346c-4518-94f5-f072f08b7c48 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:33.078298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:33.078959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:33.079235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:33.079500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:33.080287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:33.080588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:33.082025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91410442-45ac-4cfc-a3cc-0a77b269b72f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:33.084926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:33.095937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 535/1083] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:33.101906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:33.122724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:33.208601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0db0-4d6c-452d-96f9-dc743290a5c8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:33.242681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:33.243642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:33.243917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:33.244188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/bb0c5d919d5c41899857e7f333cda63e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:33.246422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=0faa6a54416f420bbdec92832d404efd, role_id=bb0c5d919d5c41899857e7f333cda63e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:33.247839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e01535-1683-4f83-b858-eda36ad57e30 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:33.311679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 549/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:31:33 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/0faa6a54416f420bbdec92832d404efd/roles/bb0c5d919d5c41899857e7f333cda63e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:31:34.558444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df31f-4371-4dce-b970-4639075c0b93 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:34.558959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df31f-4371-4dce-b970-4639075c0b93 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:34.558959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df31f-4371-4dce-b970-4639075c0b93 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:34.560302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 536/1085] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:34.565500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:34.565880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:34.566181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:34.583287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:34.584478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:34.650246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 550/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:34.672650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df31f-4371-4dce-b970-4639075c0b93 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:34.772693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df31f-4371-4dce-b970-4639075c0b93 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:34.804913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:34.806359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:34.806638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:34.806904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:34.816861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:34.824859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:34.826007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:34.827118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:34.829212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 537/1087] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:34.832000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:34.840445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:34.929756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f318f8-ef02-41b0-8f67-cc2dbe39d0d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:34.963796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:34.964463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:34.964745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:34.965310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:34.966052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:34.966357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:34.967709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:34.975787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 551/1088] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:34.982080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.036958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.080964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388d11-06a4-42f4-8edd-273b19c6d480 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.114278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.115140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.115421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.115698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] PATH_INFO: `/v3/users/3d1c5878a0cd443db3ea7e8813f421b5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.116869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] RBAC: Authorizing `identity:get_user(user_id=3d1c5878a0cd443db3ea7e8813f421b5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.118942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cd752-0e58-48df-9395-87cd5358ec1e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.120563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 538/1089] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:34 2022] GET /identity/v3/users/3d1c5878a0cd443db3ea7e8813f421b5 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:35.126393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.195292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.229010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a2a84-7c16-477c-be63-0eb0f6a4405b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.261714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.262354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.262633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.262906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.302734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e306a75-6d8d-414f-afaf-ab2f1c9f89f7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.303025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e306a75-6d8d-414f-afaf-ab2f1c9f89f7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.303310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e306a75-6d8d-414f-afaf-ab2f1c9f89f7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.304680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 539/1090] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:31:35.309974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.310263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.310542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.327466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:35.328666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:35.329910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.331414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.394060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.395692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 540/1091] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:31:35.418534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.432265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 552/1092] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:35.433297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.474221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.519589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f925-73ff-4f43-b0f4-0662ba4ec3a3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.530547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778642-8d46-4d6d-98ad-eb68d3a58bf4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.552616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.553455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.553735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.554004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.564023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.564369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:35.564894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.565187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.565464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.566635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.568076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.569805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 553/1093] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:35.573063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:35.574100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:35.574974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 541/1094] 10.222.0.47 () {56 vars in 1120 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:35.576679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.580253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.610563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.676788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd94c0e-4746-4735-929c-262b51f24154 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.681287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed764-5e8a-4576-9b96-587a8d5a6b1e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.694563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.712010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.712810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.713132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.713408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.713916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.714233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=a28ab568f5584dcd98e1232dfd0dc445, user.id=3d1c5878a0cd443db3ea7e8813f421b5) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:35.714548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.714639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.715120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.715360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.716097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:35.716294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.716387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.717743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.722262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 554/1095] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/role_assignments?user.id=3d1c5878a0cd443db3ea7e8813f421b5&scope.project.id=a28ab568f5584dcd98e1232dfd0dc445&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:35.732015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 542/1096] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:31:35.738058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:35.840633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:35.843720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0167f-11de-4a49-beef-43496133ca16 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:35.879929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:35.880648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:35.881077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:35.881460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:35.892112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:35.893588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:35.910633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:35.911611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:35.912545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:35.912545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:35.912545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:35.912545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:35.912545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 555/1097] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:35.918634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.016108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f99789-1a19-4c83-9ea6-7c0df29abc07 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.049067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:36.049721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:36.049995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:36.050261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:36.050978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:36.051276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:36.052619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:36.064362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 543/1098] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:36.070318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.171572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4eedf-fc09-4285-afe3-d5c2dc3dacdd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.206344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:36.207134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:36.207413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:36.207682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:36.215549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:36.216970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:31:36.230611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:36.231741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:36.232695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 556/1099] 10.222.0.47 () {56 vars in 1093 bytes} [Sat Jul 16 12:31:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:36.237600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.271987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:36.335169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ed01c-a5d1-4caa-9704-c333d8aeb10e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.367298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:36.368557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:36.368879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:36.369184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:36.370006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:36.370309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:36.371719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:36.388949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 544/1100] 10.222.0.47 () {56 vars in 1133 bytes} [Sat Jul 16 12:31:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:36.394683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.409485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:36.491591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32239c5-7c23-4c7c-b2b3-4d0c590b50bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.524241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:36.525160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:36.525441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:36.525712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:36.590325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:36.591757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:36.604540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:36.687660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 557/1101] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:36.693485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.787235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:36.794611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d8e8b-1a20-46f0-b911-06479f6d0c90 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.829279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:36.830063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:36.830339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:36.830608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:36.841205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:36.842627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:31:36.861583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:36.862632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:36.863546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:31:36.863546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:31:36.863546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:31:36.863546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:36.863546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 545/1102] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:31:36.869246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:36.965961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886d671-c098-4d37-a809-b49280e291d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:36.979484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:36.999803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:37.000456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.000735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.001008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.001754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:37.002057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:37.003408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:37.015422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 558/1103] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:37.020934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:37.117107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538115d-f729-40f6-aad2-038ab5d25840 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:37.151277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:37.152052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.152329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.152599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.159231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:37.160655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:31:37.174933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:37.176040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:31:37.177127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:31:37.182482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 546/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:37.183966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:37.280548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522810b0-ff2a-4c57-b19b-bbfbd1cb6be1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:37.313897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:37.314545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.314820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.315086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.315859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:37.316163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:37.317609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6f2b0-bb7c-4933-a518-650c68316370 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:37.329587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 559/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:37.335763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:37.345514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:37.435115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9952114-5f0b-4223-916d-39cb2bfb3862 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:37.467553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:37.468205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.468484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.468760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.469619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] RBAC: Adding query filter params (role.id=bb0c5d919d5c41899857e7f333cda63e, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=0faa6a54416f420bbdec92832d404efd) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:37.469928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:37.471339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74571f8a-e97b-4ab4-8f11-1bbb3be4ecd6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:37.477085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 547/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:37 2022] GET /identity/v3/role_assignments?user.id=0faa6a54416f420bbdec92832d404efd&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=bb0c5d919d5c41899857e7f333cda63e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:37.779005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6de3-0589-473e-9226-6ae35b8be11b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.779507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6de3-0589-473e-9226-6ae35b8be11b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.779507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6de3-0589-473e-9226-6ae35b8be11b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.780842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 560/1107] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:37.786346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:37.786762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:37.786907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:37.804499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:37.805729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:37.869240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:37.874048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 548/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:37.899640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df6de3-0589-473e-9226-6ae35b8be11b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:37.932234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:38.007822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df6de3-0589-473e-9226-6ae35b8be11b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:38.042197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:38.042905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:38.043184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:38.043456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:38.044638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:38.046107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf6c5-c21b-4bbc-a96b-a5f7ce78dbe1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:38.047728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 561/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:38.053775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:38.157415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4485fa-bc80-4da3-a702-f886d8a2ae10 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:38.193788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:38.194469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:38.194755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:38.195029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:38.195751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:38.197264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7615a7-1cd1-4756-8679-5b23a20016d7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:38.300131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 549/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:31:38 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:31:40.330913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17654290-5994-4eb2-974f-03adab2c00ee None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:40.331434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17654290-5994-4eb2-974f-03adab2c00ee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:40.331526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17654290-5994-4eb2-974f-03adab2c00ee None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:40.332823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 562/1111] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:40.338862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912475-258e-41df-ae51-29414a9a919d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:40.339258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912475-258e-41df-ae51-29414a9a919d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:40.339410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4912475-258e-41df-ae51-29414a9a919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:40.356076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b4912475-258e-41df-ae51-29414a9a919d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:40.357273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b4912475-258e-41df-ae51-29414a9a919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:40.409813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:40.425657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 550/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:40.448959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17654290-5994-4eb2-974f-03adab2c00ee None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:40.546321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:40.549949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17654290-5994-4eb2-974f-03adab2c00ee None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:40.583262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:40.584105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:40.584386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:40.584658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] PATH_INFO: `/v3/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:40.586147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] RBAC: Authorizing `identity:get_role(role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:40.587618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:40.588977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 563/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:40 2022] GET /identity/v3/roles/99bae25deed04db6ab077873f28587b2 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:40.595179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4912475-258e-41df-ae51-29414a9a919d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:40.690306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4912475-258e-41df-ae51-29414a9a919d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:40.723511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:40.724311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:40.724591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:40.724867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] PATH_INFO: `/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:40.737604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] RBAC: Authorizing `identity:get_user(user_id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:40.739030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:40.740672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 551/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:40 2022] GET /identity/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:40.746914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:40.844669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de839b-881a-4e5c-9ce6-0c9e76a09a12 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:40.877273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:40.877903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:40.878182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:40.878453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:40.912179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:40.946162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:40.948122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:41.042583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 564/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:41.048355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:41.094055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:41.145299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279082e-527b-4d41-83ad-3987b596c273 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:41.178952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:41.180360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:41.180360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:41.180633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:41.181818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:41.183241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60633a1-a4fc-4454-be1c-a2eebb0c4afc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:41.184946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 552/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:41 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:41.191045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:41.257928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:41.289666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1af4-4ecc-4e67-93e8-9552d4e628b7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:41.322019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:41.322601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:41.322877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:41.323145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:41.323952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=a28ab568f5584dcd98e1232dfd0dc445, user.id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:41.324259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:41.325701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7ecf8-72ab-44a3-a0d8-9416cb0ada13 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:41.330896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 565/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:41 2022] GET /identity/v3/role_assignments?user.id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd&scope.project.id=a28ab568f5584dcd98e1232dfd0dc445&role.id=99bae25deed04db6ab077873f28587b2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:43.284206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c5d89-95c5-4c2f-b45f-9aab97fedd39 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.284769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c5d89-95c5-4c2f-b45f-9aab97fedd39 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.284769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c5d89-95c5-4c2f-b45f-9aab97fedd39 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.286169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 553/1118] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:43.291866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.292311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.292413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.308519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:43.309719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:43.373005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 566/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:43.395549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c5d89-95c5-4c2f-b45f-9aab97fedd39 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:43.494625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c5d89-95c5-4c2f-b45f-9aab97fedd39 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:43.507247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:43.528420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:43.529289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.529574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.529851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] PATH_INFO: `/v3/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.531419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] RBAC: Authorizing `identity:get_role(role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:43.532685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:43.534067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 554/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:43 2022] GET /identity/v3/roles/99bae25deed04db6ab077873f28587b2 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:43.539805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:43.597967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:43.635346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c579ef9-6ee8-485b-8ae2-e8022865ab7a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:43.667067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:43.667924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.668204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.668474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] PATH_INFO: `/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.669660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] RBAC: Authorizing `identity:get_user(user_id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:43.671070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:43.672668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 567/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:43 2022] GET /identity/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:43.678340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:43.776278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8ba5e-c76b-432f-aef2-d08e98ad6f38 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:43.808560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:43.809420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.809699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.809968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.811122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:43.812526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeb1c8-59dc-4490-8a86-7f1112a60af8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:43.814222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 555/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:43 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:43.820049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:43.915674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a3e9-a31f-4d68-b068-fab2aad9f638 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:43.942998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:43.948565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:43.960223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:43.960223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:43.960223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:43.960223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] RBAC: Authorizing `identity:create_grant(project_id=a28ab568f5584dcd98e1232dfd0dc445, user_id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd, role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:43.960223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fe8a0-a1fa-4f65-a0a9-df2fdae8a274 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:44.030419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 568/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:31:43 2022] PUT /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd/roles/99bae25deed04db6ab077873f28587b2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:31:46.014020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738dcf3-b8dd-48c4-959e-2351cbca9743 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.014577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738dcf3-b8dd-48c4-959e-2351cbca9743 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.014577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738dcf3-b8dd-48c4-959e-2351cbca9743 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.015938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 556/1124] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:46.021025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.021455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.021614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.038473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:46.040256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:46.103418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 569/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:46.125988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738dcf3-b8dd-48c4-959e-2351cbca9743 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:46.175882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:46.226437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738dcf3-b8dd-48c4-959e-2351cbca9743 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:46.259131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:46.259969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.260251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.260520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] PATH_INFO: `/v3/roles/99bae25deed04db6ab077873f28587b2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.261998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] RBAC: Authorizing `identity:get_role(role_id=99bae25deed04db6ab077873f28587b2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:46.263349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:46.264746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 557/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:46 2022] GET /identity/v3/roles/99bae25deed04db6ab077873f28587b2 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:46.270572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:46.364987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b84772-b12c-4a47-9e85-c7ad9184fdfe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:46.384989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:46.398068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:46.399562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.399835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.400102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] PATH_INFO: `/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.401295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] RBAC: Authorizing `identity:get_user(user_id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:46.402702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:46.404286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 570/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:46 2022] GET /identity/v3/users/b6cd2aaea81a4ee5bf2d6db8cbffc1dd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:46.410025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:46.504776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b379-2689-4cfe-bbb7-3e6989ecbc2c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:46.521209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:46.538758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:46.539588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.539859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.540129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.605703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:46.607127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:46.702443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 558/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 12:31:46.708248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:46.802393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188758-4420-4ffd-9d2e-b3d95454a234 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:46.835018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:46.835852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.836311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.836714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] PATH_INFO: `/v3/projects/a28ab568f5584dcd98e1232dfd0dc445` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.837893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] RBAC: Authorizing `identity:get_project(project_id=a28ab568f5584dcd98e1232dfd0dc445)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:46.839551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793cfde-0072-4b70-834d-0212928d1262 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:46.841272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 571/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:46 2022] GET /identity/v3/projects/a28ab568f5584dcd98e1232dfd0dc445 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:46.847412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:46.864277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:46.943418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c67d8-0086-483a-98d7-9e67b451b7c3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:46.976188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:46.977391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:46.977753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:46.978028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:46.978853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] RBAC: Adding query filter params (role.id=99bae25deed04db6ab077873f28587b2, scope.project.id=a28ab568f5584dcd98e1232dfd0dc445, user.id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:46.979162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:46.980575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571dbf8-cca0-4e39-bdb8-a8d1d87d4d63 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:46.986228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 559/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:46 2022] GET /identity/v3/role_assignments?user.id=b6cd2aaea81a4ee5bf2d6db8cbffc1dd&scope.project.id=a28ab568f5584dcd98e1232dfd0dc445&role.id=99bae25deed04db6ab077873f28587b2 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:49.017252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830da2-4032-43c4-b7e1-8e018d696477 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:49.017778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830da2-4032-43c4-b7e1-8e018d696477 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:49.017778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60830da2-4032-43c4-b7e1-8e018d696477 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:49.019193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 572/1131] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:49.025142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:49.025548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:49.025695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:49.042415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:49.043578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:49.115879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 560/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:49.138663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830da2-4032-43c4-b7e1-8e018d696477 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:49.194706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:49.238089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60830da2-4032-43c4-b7e1-8e018d696477 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:49.270488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:49.271209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:49.271491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:49.271763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:49.272927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:49.274998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d512fda-b1cb-41ec-8405-7696303aece1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:49.276639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 573/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:49.282723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:49.294286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:49.378844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d062774-5b6b-4e5a-bea8-fb4b41094fb4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:49.412134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:49.412768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:49.413065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:49.413339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:49.414219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:49.415772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e4969-9ae0-4087-80c9-2b0d07d377ff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:49.493937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 561/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:31:49 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:51.552080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fbeba-409d-4e59-b3dc-5f86385cd59d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:51.552600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fbeba-409d-4e59-b3dc-5f86385cd59d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:51.552600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fbeba-409d-4e59-b3dc-5f86385cd59d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:51.553935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 574/1135] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:51.559132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:51.559556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:51.559702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:51.576522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:51.577727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:51.634642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:51.651221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 562/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:51.674393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fbeba-409d-4e59-b3dc-5f86385cd59d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:51.780164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fbeba-409d-4e59-b3dc-5f86385cd59d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:51.805838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:51.818479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:51.819265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:51.819553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:51.820057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:51.822234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:51.823189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674786d0-2250-4b41-aa39-da60466f7694 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:51.824821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 575/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:51.830728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:51.936811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334cff2-0cbe-405f-a238-c4297d00eb20 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:51.974522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:51.975187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:51.975548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:51.975827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:51.977413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:51.978855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9416e-5a46-4dfa-aa0a-bde8ce6ab051 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:52.053617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 563/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:31:51 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:31:54.123640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30dcb-e693-4b79-91ed-13006a16d775 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.124175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30dcb-e693-4b79-91ed-13006a16d775 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.124175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30dcb-e693-4b79-91ed-13006a16d775 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.125637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 576/1139] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:54.130694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.131132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.131235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.148351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:54.149678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:54.192731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:54.215716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 564/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:54.238887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30dcb-e693-4b79-91ed-13006a16d775 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:54.338058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30dcb-e693-4b79-91ed-13006a16d775 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:54.370702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:54.371474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.371757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.372032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.381819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:54.390129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:54.391295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:54.392270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 577/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:54.397303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:54.493180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51046596-0d4d-4052-894b-6094ef7009fe None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:54.525115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:54.525758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.526032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.526299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.527024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:54.527324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:54.528665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:54.536327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 565/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:54.542499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:54.593510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:54.639921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce97c-f712-4cb1-8cf2-675fcce7b923 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:54.672206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:54.673080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.673365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.673641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] PATH_INFO: `/v3/users/2efc447861c24cf8add85bf3f48d4675` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.687604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] RBAC: Authorizing `identity:get_user(user_id=2efc447861c24cf8add85bf3f48d4675)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:54.689026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:54.690718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 578/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:54 2022] GET /identity/v3/users/2efc447861c24cf8add85bf3f48d4675 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:54.696851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:54.746610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:54.803033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d05de-6277-41b9-b491-8a08eae9438f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:54.834697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:54.835840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:54.836170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:54.836438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:54.901565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:54.902933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:54.941051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:54.998722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 566/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:31:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:31:55.004774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:55.103669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189ba5-096e-4ca8-90e3-9a6a30b541ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:55.122047 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:55.144636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:55.145515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:55.145792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:55.146061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] PATH_INFO: `/v3/projects/18644ee9a7934ce5942d4967e039b911` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:55.147212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=18644ee9a7934ce5942d4967e039b911)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:55.149731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80db30-2093-42e0-9b47-ef75d7fa0ae9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:55.151409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 579/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:55 2022] GET /identity/v3/projects/18644ee9a7934ce5942d4967e039b911 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:55.157668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:55.254848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f74f37-64cd-44db-b55d-5b69588f6506 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:55.288432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:55.289549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:55.289828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:55.290097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:55.290911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=18644ee9a7934ce5942d4967e039b911, user.id=2efc447861c24cf8add85bf3f48d4675) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:55.291307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:55.292709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22578-0fa9-4ff2-96a9-526eb8e059bf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:55.298079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 567/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:31:55 2022] GET /identity/v3/role_assignments?user.id=2efc447861c24cf8add85bf3f48d4675&scope.project.id=18644ee9a7934ce5942d4967e039b911&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:57.285503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b001a-cd8d-42a3-8f31-2615c9d39d22 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.286018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b001a-cd8d-42a3-8f31-2615c9d39d22 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.286156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b001a-cd8d-42a3-8f31-2615c9d39d22 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.287446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 580/1147] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:31:57.292723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.310639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.310639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.310639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:57.310639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:31:57.311696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:31:57.377088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 568/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:31:57.399895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b001a-cd8d-42a3-8f31-2615c9d39d22 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:57.500273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b001a-cd8d-42a3-8f31-2615c9d39d22 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:57.532185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:57.532945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.533251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.533522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.544022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:31:57.551420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:31:57.552513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:31:57.553617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:31:57.557187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 581/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:31:57.559353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:57.654919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb655a8-d711-4485-9829-f9b2280b2699 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:57.687546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:57.688201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.688486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.688764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.689531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:31:57.689840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:57.691193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:57.692526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:57.700174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 569/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:57.706309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:57.747221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:57.805863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3070cf-6958-4f1f-8e1e-8e247d7e0399 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:57.838140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:57.839246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.839524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.839794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] PATH_INFO: `/v3/users/2efc447861c24cf8add85bf3f48d4675` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.840953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] RBAC: Authorizing `identity:get_user(user_id=2efc447861c24cf8add85bf3f48d4675)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:57.842400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:57.844005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 582/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:31:57 2022] GET /identity/v3/users/2efc447861c24cf8add85bf3f48d4675 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:57.849711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:57.947938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c640a-186c-47ef-be49-1e4a0fc37caa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:57.980204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:57.981075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:57.981363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:57.981636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] PATH_INFO: `/v3/projects/18644ee9a7934ce5942d4967e039b911` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:57.982813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] RBAC: Authorizing `identity:get_project(project_id=18644ee9a7934ce5942d4967e039b911)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:57.984243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5a8ee-5f2d-44d2-836a-e329c7a26b51 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:57.985956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 570/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:31:57 2022] GET /identity/v3/projects/18644ee9a7934ce5942d4967e039b911 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:31:57.992790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:31:58.092797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60f83b-1f14-4004-b21b-1d4c131f096a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:31:58.097169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:31:58.124538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:31:58.125445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:31:58.125726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:31:58.125998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] PATH_INFO: `/v3/projects/18644ee9a7934ce5942d4967e039b911/users/2efc447861c24cf8add85bf3f48d4675/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:31:58.127916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] RBAC: Authorizing `identity:create_grant(project_id=18644ee9a7934ce5942d4967e039b911, user_id=2efc447861c24cf8add85bf3f48d4675, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:31:58.129363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a84c05-02ae-4511-9eb7-614b64798aca None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:31:58.201523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 583/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:31:57 2022] PUT /identity/v3/projects/18644ee9a7934ce5942d4967e039b911/users/2efc447861c24cf8add85bf3f48d4675/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:32:00.161670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417512-13f2-44d5-bd04-ebf9375ad00f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.162229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417512-13f2-44d5-bd04-ebf9375ad00f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.162229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417512-13f2-44d5-bd04-ebf9375ad00f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.163721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 571/1154] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:00.168962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.169370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.169547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.186622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:00.187804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:00.251518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 584/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:00.275798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5417512-13f2-44d5-bd04-ebf9375ad00f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:00.337738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:00.375400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5417512-13f2-44d5-bd04-ebf9375ad00f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:00.408050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:00.408823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.409130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.409403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.420001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:32:00.427516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:32:00.428661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:32:00.429961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 572/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:32:00.434904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:00.525201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:00.530680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d71f-c883-4971-8459-819a734d4d59 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:00.564028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:00.564679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.566413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.566413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.566413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:00.566413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:00.567594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:00.575887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 585/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:00.581696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:00.680341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7063c-9428-480a-9c22-e86e7843d663 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:00.702710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:00.714039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:00.714901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.715181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.715454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] PATH_INFO: `/v3/users/2efc447861c24cf8add85bf3f48d4675` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.716630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] RBAC: Authorizing `identity:get_user(user_id=2efc447861c24cf8add85bf3f48d4675)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:00.718078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:00.719971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 573/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:32:00 2022] GET /identity/v3/users/2efc447861c24cf8add85bf3f48d4675 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:00.725490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:00.822978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299428b2-2912-4c84-952c-cfa21ed6b162 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:00.855153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:00.855795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:00.856084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:00.856356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:00.883116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:00.922180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:00.923608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:01.017996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 586/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:32:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:32:01.024088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:01.121099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ccc51-719d-4136-a6bc-f6ee390f8344 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:01.153139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:01.154013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:01.154288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:01.154558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] PATH_INFO: `/v3/projects/18644ee9a7934ce5942d4967e039b911` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:01.155698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] RBAC: Authorizing `identity:get_project(project_id=18644ee9a7934ce5942d4967e039b911)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:01.157126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cf9a3-a3e3-4007-b24f-785ebb9a53fb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:01.158784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 574/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:32:01 2022] GET /identity/v3/projects/18644ee9a7934ce5942d4967e039b911 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:01.165511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:01.220846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:01.261397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb00c9-e8b8-4475-adf0-0d11b7d45f7b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:01.293376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:01.293953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:01.294230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:01.294500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:01.295312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=18644ee9a7934ce5942d4967e039b911, user.id=2efc447861c24cf8add85bf3f48d4675) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:01.295617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:01.297024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9dfd1-4c83-4eb6-bdeb-4390f7049fd7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:01.302594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 587/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:32:01 2022] GET /identity/v3/role_assignments?user.id=2efc447861c24cf8add85bf3f48d4675&scope.project.id=18644ee9a7934ce5942d4967e039b911&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:03.313519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2987e6-f20c-4fff-839d-85def3b76e85 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:03.314024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2987e6-f20c-4fff-839d-85def3b76e85 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:03.314147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2987e6-f20c-4fff-839d-85def3b76e85 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:03.315428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 575/1162] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:03.320706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:03.321141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:03.321303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:03.338143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:03.339601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:03.403796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 588/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:03.426206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2987e6-f20c-4fff-839d-85def3b76e85 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:03.485257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:03.525699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2987e6-f20c-4fff-839d-85def3b76e85 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:03.558185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:03.558820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:03.559102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:03.559379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:03.624626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:03.626092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:03.680897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:03.721686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 576/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:32:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:32:03.728125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:03.779396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:03.825252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3138a69-7e32-4ba1-982a-c0dd34d713a0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:03.857561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:03.858338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:03.858619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:03.858893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:03.865313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:03.867364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ca7821b8-6f31-43c3-b8ee-10d43e2d2d19 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:32:03.880846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:32:03.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:32:03.883747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 589/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:32:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:32:03.888880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:03.982754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299c77b-34e8-4c9e-858d-2d5794673f17 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:04.015761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:04.016406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:04.016683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:04.016958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:04.017770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:04.018074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:04.019424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07aeea-25fc-4c8d-a97e-82b8feb899b0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:04.026361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 577/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Sat Jul 16 12:32:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:05.996705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347791-fc36-4b15-b08c-358c2e45b3af None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:05.997299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347791-fc36-4b15-b08c-358c2e45b3af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:05.997407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8347791-fc36-4b15-b08c-358c2e45b3af None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:05.998717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 590/1167] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:06.003915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d670-6448-4147-a3e3-2d127ba0fd29 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:06.004288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d670-6448-4147-a3e3-2d127ba0fd29 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:06.004471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d670-6448-4147-a3e3-2d127ba0fd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:06.021272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b950d670-6448-4147-a3e3-2d127ba0fd29 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:06.023044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b950d670-6448-4147-a3e3-2d127ba0fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:06.039737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:06.087716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 578/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:06.110241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8347791-fc36-4b15-b08c-358c2e45b3af None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:06.210026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8347791-fc36-4b15-b08c-358c2e45b3af None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:06.222685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:06.243236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:06.243910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:06.244216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:06.244487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:06.245246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:06.246597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a8e1d-6624-4b7a-92eb-b97c067b1f5e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:06.310777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 591/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Sat Jul 16 12:32:06 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:32:08.320717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52a23-e062-4c11-970a-c8ede14b5155 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:08.321302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52a23-e062-4c11-970a-c8ede14b5155 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:08.321302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52a23-e062-4c11-970a-c8ede14b5155 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:08.322653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 579/1170] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:08.327890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:08.328278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:08.328442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:08.345399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:08.346555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:08.410783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 592/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:08.435041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52a23-e062-4c11-970a-c8ede14b5155 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:08.532457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52a23-e062-4c11-970a-c8ede14b5155 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:08.564784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:08.565643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:08.565921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:08.566189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] PATH_INFO: `/v3/domains/93184f10e0fd4879a8e434655fb8d73c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:08.576298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93184f10e0fd4879a8e434655fb8d73c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:08.577749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b56d8f-07ba-444b-a292-9d7cb94dfc96 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:08.579346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 580/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Sat Jul 16 12:32:08 2022] GET /identity/v3/domains/93184f10e0fd4879a8e434655fb8d73c => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:08.586046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:08.684137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf7ca9-f773-4814-b2eb-638015d9c668 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:08.685814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:08.727748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:08.728381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:08.728659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:08.728931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:08.729651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:08.731069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7a53a-21d4-4157-afed-c112075c16f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:08.801283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 593/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:32:08 2022] POST /identity/v3/projects => generated 361 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:32:10.835886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081de9-c40c-4be5-80aa-9fc78cf5973b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:10.836415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081de9-c40c-4be5-80aa-9fc78cf5973b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:10.836415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081de9-c40c-4be5-80aa-9fc78cf5973b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:10.838050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 581/1174] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:10.843256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:10.843681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:10.843817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:10.860866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:10.862072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:10.925174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 594/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:10.949189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081de9-c40c-4be5-80aa-9fc78cf5973b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:10.965677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:11.048051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081de9-c40c-4be5-80aa-9fc78cf5973b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:11.080569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:11.081446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:11.081727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:11.081999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] PATH_INFO: `/v3/domains/93184f10e0fd4879a8e434655fb8d73c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:11.083156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93184f10e0fd4879a8e434655fb8d73c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:11.085280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd44104-b9db-4faf-af8d-8f24d818bfa6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:11.086926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 582/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Sat Jul 16 12:32:10 2022] GET /identity/v3/domains/93184f10e0fd4879a8e434655fb8d73c => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:11.092890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:11.172746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:11.187819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292741-67dd-4c88-917c-cdfa88c075bf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:11.222152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:11.222810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:11.223094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:11.223369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:11.224088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:11.225720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78890c26-3332-44b0-8d13-6980b1cbc3f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:11.303593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 595/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:32:11 2022] POST /identity/v3/users => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:32:13.347419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e3a8-fd76-468e-bff2-5d8af4872b59 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:13.347992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e3a8-fd76-468e-bff2-5d8af4872b59 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:13.347992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e3a8-fd76-468e-bff2-5d8af4872b59 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:13.349478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 583/1178] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:13.355325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:13.355720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:13.355879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:13.373410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:13.374580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:13.440209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 596/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:13.463395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e3a8-fd76-468e-bff2-5d8af4872b59 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:13.545131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:13.563022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e3a8-fd76-468e-bff2-5d8af4872b59 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:13.595801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:13.597679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:13.597679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:13.597679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:13.607477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:32:13.614946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:32:13.616085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:32:13.617143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 584/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:32:13.622444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:13.688427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:13.716781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47643a1a-563d-4787-b9aa-9187a0d01660 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:13.748946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:13.749615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:13.749893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:13.750163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:13.750891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:13.751190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:13.752531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:13.760668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 597/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:13.766607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:13.864628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ad844-f078-439d-9cf9-56fc24919e0d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:13.896303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:13.897720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:13.898001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:13.898272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] PATH_INFO: `/v3/users/665e3ae79d1d48c68e4d62d69bac79f0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:13.908915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:13.911302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=665e3ae79d1d48c68e4d62d69bac79f0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:13.913378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:13.915008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 585/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:32:13 2022] GET /identity/v3/users/665e3ae79d1d48c68e4d62d69bac79f0 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:13.920848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:14.017921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811943a-872a-445c-9e3b-18dcbf25903a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:14.046778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:14.051352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:14.051990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:14.052275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:14.052549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:14.117409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:14.118844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:14.214837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 598/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:32:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:32:14.220735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:14.317198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee56a5-4a3c-4aa5-af9b-0fdb51e9a6e6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:14.350658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:14.351497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:14.351780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:14.352054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] PATH_INFO: `/v3/projects/7f35fa053a8c48ffbe023ee075d9967c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:14.353254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] RBAC: Authorizing `identity:get_project(project_id=7f35fa053a8c48ffbe023ee075d9967c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:14.354668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cedf0-2713-4a3d-ac1a-2ce7bfef721a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:14.356333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 586/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:32:14 2022] GET /identity/v3/projects/7f35fa053a8c48ffbe023ee075d9967c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:14.362349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:14.392463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:14.457690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf288edf-26e1-4da4-9c58-9bce27366764 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:14.491631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:14.492267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:14.492550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:14.492824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:14.493678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=7f35fa053a8c48ffbe023ee075d9967c, user.id=665e3ae79d1d48c68e4d62d69bac79f0) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:14.493993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:14.495405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56511fa-00c4-4fbf-aa4b-ab50b4e9fc07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:14.500705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 599/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:32:14 2022] GET /identity/v3/role_assignments?user.id=665e3ae79d1d48c68e4d62d69bac79f0&scope.project.id=7f35fa053a8c48ffbe023ee075d9967c&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:16.487755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031925-5068-4f57-a668-8f8ddc26bc74 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:16.488270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031925-5068-4f57-a668-8f8ddc26bc74 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:16.488270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031925-5068-4f57-a668-8f8ddc26bc74 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:16.489619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 587/1186] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:16.494809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:16.495223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:16.495354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:16.512121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:16.513330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:16.576962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 600/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 12:32:16.599531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031925-5068-4f57-a668-8f8ddc26bc74 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:16.665614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:16.699609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031925-5068-4f57-a668-8f8ddc26bc74 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:16.733175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:16.733950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:16.734235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:16.734512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:16.745093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:32:16.752743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:32:16.753763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:32:16.754656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 588/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:32:16.760708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:16.787349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:16.858054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f856d1-a830-4b82-92d2-0f8fbfcc174a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:16.890827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:16.891472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:16.891750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:16.892020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:16.892757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:16.893082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:16.894445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:16.902321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 601/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:16.908137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:17.009815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377d14-e404-4ab7-8fcd-1a2286600907 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:17.034438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:17.042925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] PATH_INFO: `/v3/users/665e3ae79d1d48c68e4d62d69bac79f0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] RBAC: Authorizing `identity:get_user(user_id=665e3ae79d1d48c68e4d62d69bac79f0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:17.048642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 589/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:32:16 2022] GET /identity/v3/users/665e3ae79d1d48c68e4d62d69bac79f0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:17.055025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:17.138821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:17.150214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c062d3f-d0b9-4864-b29b-c12af6ee57c1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:17.184689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:17.185578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:17.185862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:17.186137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] PATH_INFO: `/v3/projects/7f35fa053a8c48ffbe023ee075d9967c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:17.187298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] RBAC: Authorizing `identity:get_project(project_id=7f35fa053a8c48ffbe023ee075d9967c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:17.188719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e32adb-4aa5-45a1-beae-2e25c7a3ade3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:17.190454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 602/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:32:17 2022] GET /identity/v3/projects/7f35fa053a8c48ffbe023ee075d9967c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:17.196426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:17.294623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5a27c-6273-46e4-a686-bfd74ad615fc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:17.326154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:17.327017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:17.327293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:17.327562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] PATH_INFO: `/v3/projects/7f35fa053a8c48ffbe023ee075d9967c/users/665e3ae79d1d48c68e4d62d69bac79f0/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:17.329487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] RBAC: Authorizing `identity:create_grant(project_id=7f35fa053a8c48ffbe023ee075d9967c, user_id=665e3ae79d1d48c68e4d62d69bac79f0, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:17.330882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f49ec-f1b1-45ae-929e-6da005a5594e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:17.405597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 590/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:32:17 2022] PUT /identity/v3/projects/7f35fa053a8c48ffbe023ee075d9967c/users/665e3ae79d1d48c68e4d62d69bac79f0/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:32:19.424677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e2f0-645a-49c6-b598-4974daa5e326 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:19.425219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e2f0-645a-49c6-b598-4974daa5e326 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:19.425219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e2f0-645a-49c6-b598-4974daa5e326 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:19.426607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 603/1193] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:32:19.431868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:19.432274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:19.432372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:19.449023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:32:19.450229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:32:19.497812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:19.513939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 591/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:32:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:32:19.536910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19e2f0-645a-49c6-b598-4974daa5e326 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:19.636204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19e2f0-645a-49c6-b598-4974daa5e326 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:19.668643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:19.669439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:19.669719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:19.669992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:19.680350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:32:19.687927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:32:19.689014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:32:19.690095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 604/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:32:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:32:19.695744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:19.789883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f56f1-4bf9-4b73-a7b0-98ef7a1d1271 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:19.822809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:19.823459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:19.823738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:19.824009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:19.824740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:19.825068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:19.826409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:19.834360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 592/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:32:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:19.840182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:19.858055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:19.935884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3391e-5c39-4d21-9793-739eb7966287 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:19.968471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:19.969294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:19.969579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:19.969854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] PATH_INFO: `/v3/users/665e3ae79d1d48c68e4d62d69bac79f0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:19.971018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] RBAC: Authorizing `identity:get_user(user_id=665e3ae79d1d48c68e4d62d69bac79f0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:19.972434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:19.974072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 605/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Sat Jul 16 12:32:19 2022] GET /identity/v3/users/665e3ae79d1d48c68e4d62d69bac79f0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:19.980531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:20.027925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:20.077932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f543cd-3bf5-46bb-b54d-c9ce95f6219e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:20.110088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:20.110754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:20.111035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:20.111307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:20.176247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:20.178338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:20.223710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:20.273503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 593/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:32:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:32:20.279473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:20.360327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:20.375199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffdab6-5c28-4900-a3f7-5c2b08f92a8f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:20.408333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:20.409208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:20.409492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:20.409766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] PATH_INFO: `/v3/projects/7f35fa053a8c48ffbe023ee075d9967c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:20.410926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7f35fa053a8c48ffbe023ee075d9967c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:20.412898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcce593-e4f0-4260-bc6a-5efaf98766a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:20.414624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 606/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Sat Jul 16 12:32:20 2022] GET /identity/v3/projects/7f35fa053a8c48ffbe023ee075d9967c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:20.420782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:32:20.515957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9647865-df9b-46ec-8a63-355f09344563 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:32:20.548918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:32:20.549579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:32:20.549860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:32:20.550134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:32:20.550946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=7f35fa053a8c48ffbe023ee075d9967c, user.id=665e3ae79d1d48c68e4d62d69bac79f0) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:32:20.551257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:32:20.552692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7f49-44b4-4680-8cbe-69c6c0271a39 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:32:20.558345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 594/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:32:20 2022] GET /identity/v3/role_assignments?user.id=665e3ae79d1d48c68e4d62d69bac79f0&scope.project.id=7f35fa053a8c48ffbe023ee075d9967c&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:32:30.404247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=24 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:32:30.555864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:30.865420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf41e9-ae8c-48cb-881b-1d90163e50a9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:30.866020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf41e9-ae8c-48cb-881b-1d90163e50a9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:30.866020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf41e9-ae8c-48cb-881b-1d90163e50a9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:30.867331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 607/1201] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:30.872558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:30.872942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:30.873149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:30.891130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:30.892317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:30.959038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 595/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:30.982973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf41e9-ae8c-48cb-881b-1d90163e50a9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:31.084313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf41e9-ae8c-48cb-881b-1d90163e50a9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:31.117443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:31.118159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:31.118436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:31.118706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:31.129461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:31.130895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:31.148484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:31.149554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:31.150630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:31.150630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:31.150630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:31.150630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:31.150630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 608/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:31.156024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:31.257118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea34a-ccde-4e72-9978-2adcc1d60560 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:31.284278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:31.290240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:31.290918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:31.291196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:31.291470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:31.292187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:31.292492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:31.293890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c06c3-e934-4a28-8c4c-cbc0f6997d88 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:31.305150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 596/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:33:31.310994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:31.364494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:31.416794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168de36-8c0b-4656-a38f-c23e7a2afd8e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:31.451416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:31.452580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:31.452580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:31.452757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:31.453339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:31.454765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852ffb9-7613-4fbf-b9f3-c3da9d6b773a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:31.527295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 609/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Sat Jul 16 12:33:31 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:33:33.577653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159a96-35f3-4971-908e-91e9eb123521 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:33.578169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159a96-35f3-4971-908e-91e9eb123521 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:33.578169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159a96-35f3-4971-908e-91e9eb123521 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:33.579520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 597/1206] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:33.585758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:33.586195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:33.586303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:33.604061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:33.605262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:33.671359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 610/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:33.695141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07159a96-35f3-4971-908e-91e9eb123521 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:33.796242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07159a96-35f3-4971-908e-91e9eb123521 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:33.830675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:33.832065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:33.832337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:33.832580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:33.843572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:33.851303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:33.852457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:33.853536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 598/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:33.858781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:33.898172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:33.955349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707630a-7fe2-4d79-8f15-234c13f75ec7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:33.988307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:33.988968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:33.989320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:33.989595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:33.990358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:33.990698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:33.992058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.000732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 611/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:34.006992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.056345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:34.105237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381622ba-d684-490d-9b47-eec43f264816 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:34.139390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:34.140162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:34.140441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:34.140729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:34.151334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:34.152754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:34.170462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:34.171444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:34.172380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:34.172380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:34.172380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:34.172380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:34.172380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 599/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:33:34.178787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.275015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:34.278045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e03cd-075e-4e96-ae23-58d384dbb67e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:34.310984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:34.311628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:34.311906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:34.312176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:34.312887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:34.313244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:34.314590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.325670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 612/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:34.332043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.429520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca525-894a-4713-834a-3d0f3426bd20 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:34.431029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:34.463113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:34.463978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:34.464253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:34.464523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:34.473816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:34.475360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:34.489555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:34.490705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:34.491905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 600/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:34.496659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.598231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961662-eae8-4106-95d2-0adcba169a5c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:34.634197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:34.634893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:34.635192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:34.635461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:34.636293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:34.636595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:34.638046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.656003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 613/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:34.662015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.763895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ddda0-1841-4a28-87df-cdb90f7b71e4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:34.797288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:34.797931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:34.798212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:34.798486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:34.827586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:34.865009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:34.866467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:34.963617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 601/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:33:34.969439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:34.988909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:35.068325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a362-500f-4063-b963-2a7c829822f0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:35.101875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:35.107602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:35.107602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:35.107602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:35.113129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:35.114555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:35.131834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:35.133746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:35.135366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:35.135366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:35.135366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:35.135366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:35.135366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 614/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:35.138980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:35.202080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:35.240141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0dd5a-b417-4500-a6c5-59dad7d1a499 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:35.273889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:35.274858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:35.275223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:35.275583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:35.276401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:35.276792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:35.278274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:35.290390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 602/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:35.296287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:35.371975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:35.401079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab39-67b2-45f0-bb21-058fd295b49f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:35.436734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:35.437548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:35.437833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:35.438107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:35.444722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:35.446201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:35.459793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:35.460909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:35.462023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 615/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:35.466934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:35.566572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ffdc4-f3f5-4bbf-a15d-5b53b03ccc3e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:35.588408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:35.600594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:35.601343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:35.601626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:35.601900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:35.602677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:35.602986 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:35.604400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b55dd-9d3b-4bc0-8531-d83772bd486e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:35.616638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 603/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:35.623017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:35.719498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267738e0-099e-4aed-87fb-f3a07031080b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:35.747175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:35.752870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:35.753625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:35.753997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:35.754355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:35.755212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=c73b6a617d4546599d336eeb5b45e858) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:35.755519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:35.756929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25278e9e-2ecf-4472-adcb-bc0d53b7aa26 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:35.762271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 616/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:33:35 2022] GET /identity/v3/role_assignments?user.id=c73b6a617d4546599d336eeb5b45e858&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:37.756037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e203ea-1794-4549-b489-6b4ecb75d8c5 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:37.756621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e203ea-1794-4549-b489-6b4ecb75d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:37.756621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e203ea-1794-4549-b489-6b4ecb75d8c5 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:37.758025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 604/1220] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:33:37.763260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:37.763637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:37.763811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:37.781205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:37.783819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:37.848562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 617/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:37.871385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e203ea-1794-4549-b489-6b4ecb75d8c5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:37.972486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e203ea-1794-4549-b489-6b4ecb75d8c5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.007516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.008245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.008530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.008807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.018944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:38.026545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:38.031108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:38.032146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 605/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:38.033925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.132487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df105a7-65d0-4182-bd5e-17e696bd2c70 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.155020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:38.165435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.166136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.166415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.166691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.167432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:38.167742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.169193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.178088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 618/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:38.183863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.227571 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:38.284868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288463-f58a-430e-a21e-6d89ef7fd57b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.317439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.318215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.318494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.318766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.329148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.330570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:38.348001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:38.351376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:38.352612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:38.352612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:38.352612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:38.352612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:38.352612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 606/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:38.355129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.452839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c0c8-edfb-4279-aa69-a88d68840b6d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.485420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.486113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.486401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.486680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.487978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:38.488280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.489668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.500963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 619/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:38.506827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.602002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:38.608459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3852cf-2247-45d4-9209-a2535bbcc976 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.641559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.642368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.642648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.642923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.651292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.652765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:38.668110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:38.669265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:38.670226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 607/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:33:38.675997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.723297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:38.774370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e033e97-9357-4220-9d1d-1c84c867920a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.807298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.807965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.808245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.808518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.809382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:38.809719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.811135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.827819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 620/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:38.834817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:38.933678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6823886-aa68-4ec6-bc61-75974a41ede9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:38.967302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:38.968081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:38.968365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:38.968639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:38.979726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:38.981193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:38.997613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:39.001468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:39.003098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:39.003098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:39.003098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:39.003098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:39.003098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 608/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:39.005927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:39.104000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba022-4284-47cb-a90d-732398c52678 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:39.106289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:39.138268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:39.138929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:39.139208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:39.139484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:39.140198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:39.140501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:39.141893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:39.153115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 621/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:39.160275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:39.169499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:39.258723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef3de9-8264-4eee-9d22-1656c356e3ae None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:39.292512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:39.293320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:39.293596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:39.293868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:39.301080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:39.302566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:39.317213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:39.318286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:39.319605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 609/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:39.324397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:39.423397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c4bc-e54c-4d27-97fa-6348403a0b61 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:39.456515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:39.457296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:39.457572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:39.457841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:39.458877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:39.459182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:39.460604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa02f43-df5f-44d5-9faa-12cfaa6557fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:39.473988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 622/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:39.479991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:39.545699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:39.579335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771b80f-5b41-45df-9e45-32ed745f8edc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:39.613377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:39.614691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:39.614972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:39.615246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/c73b6a617d4546599d336eeb5b45e858/roles/4ad18fd3cb934f379a61fadbc431b9af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:39.629096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=c73b6a617d4546599d336eeb5b45e858, role_id=4ad18fd3cb934f379a61fadbc431b9af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:39.630525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb32a-0668-49ad-a139-0b0d1fb55417 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:39.704205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 610/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:33:39 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/c73b6a617d4546599d336eeb5b45e858/roles/4ad18fd3cb934f379a61fadbc431b9af => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:33:41.684704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e926-ebb6-4031-b41d-5cf8ebe12d40 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:41.685274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e926-ebb6-4031-b41d-5cf8ebe12d40 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:41.685274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e926-ebb6-4031-b41d-5cf8ebe12d40 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:41.686651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 623/1233] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:41.692633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:41.693090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:41.693301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:41.709407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:41.710623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:41.776531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 611/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:41.800309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e926-ebb6-4031-b41d-5cf8ebe12d40 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:41.825298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:41.899513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e926-ebb6-4031-b41d-5cf8ebe12d40 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:41.933590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:41.934395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:41.934677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:41.934999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:41.945884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:41.953385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:41.954539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:41.955966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 624/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Sat Jul 16 12:33:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:41.961789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.009582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:42.061460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59e889-b134-4d7a-bb45-7d69393d8904 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.095615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.096277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.096556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.096832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.097647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:42.097953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.099303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:42.107484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 612/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Sat Jul 16 12:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:42.113376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.198324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:42.214170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f5f4b-b974-48fb-bfe6-ec26e345a31c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.246606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.247386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.247670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.247934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.258191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.259773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:42.278866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:42.279949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:42.288774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:42.288774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:42.288774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:42.288774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:42.288774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 625/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:42.289174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.383787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f66e6-3c62-4b1c-ba2d-51f651fe2fd3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.392784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:42.416794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.417507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.417785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.418055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.418835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:42.419137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.420480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:42.432040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 613/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:42.438005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.546234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce51a-0e83-42f9-af10-b2e8b1264b81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.579940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.581203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.581483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.581751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.584803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:42.589371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.590790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:42.605529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:42.606678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:42.608007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 626/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:42.613709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.718571 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f318792-7b99-490f-a976-6fe61815793a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.751477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.752134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.752949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.753252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.754070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:42.754402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.755798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:42.773291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 614/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:42.779048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:42.881532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4347e2-8d2c-4bc6-af4f-1b45a46afd6f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:42.914619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:42.915251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:42.915532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:42.915806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:42.976793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:42.983109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:42.984611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.080906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 627/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:33:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:33:43.087075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:43.115440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:43.186283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99964-dde9-42bc-8bc7-6041a483067b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:43.220850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:43.222296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:43.222573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:43.222864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:43.232820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:43.234294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:43.251213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:43.252309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:43.253421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:43.253421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:43.253421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:43.253421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:43.253421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 615/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:43.258237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:43.352191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:43.357618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87ae58-2c10-4107-9cab-061970ea359a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:43.390636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:43.391294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:43.391578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:43.391851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:43.392562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:43.393416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:43.394778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.406359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 628/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:43.413481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:43.498333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:43.520156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf55c-572c-471a-ac8d-7d75516ea7d9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:43.552639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:43.553501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:43.553779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:43.554053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:43.560582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:43.562033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:43.575808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:43.576917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:43.578270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 616/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:43.583714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:43.683165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d6a2c-9967-475b-afec-cbd08f047eee None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:43.716703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:43.717378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:43.717656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:43.723055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:43.723055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:43.723055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:43.723055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6f51-0194-4505-852a-aacde5e95773 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.732407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 629/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:43.739153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:43.842140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0441e-9488-42eb-9065-384cb014729e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:43.875238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:43.875878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:43.876157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:43.876430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:43.877848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] RBAC: Adding query filter params (role.id=4ad18fd3cb934f379a61fadbc431b9af, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=c73b6a617d4546599d336eeb5b45e858) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:43.878158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:43.879680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2ed0a-bebb-40d4-9fbc-236ff6560fb8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:43.885386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 617/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:33:43 2022] GET /identity/v3/role_assignments?user.id=c73b6a617d4546599d336eeb5b45e858&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=4ad18fd3cb934f379a61fadbc431b9af => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:45.931397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd79f8-8467-43db-a869-9af85bab5f16 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:45.931998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd79f8-8467-43db-a869-9af85bab5f16 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:45.931998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd79f8-8467-43db-a869-9af85bab5f16 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:45.933322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 630/1247] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:45.938513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:45.938949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:45.939102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:45.944266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:45.956350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:45.957909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:46.023675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 618/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:46.047103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd79f8-8467-43db-a869-9af85bab5f16 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.065589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:46.148043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd79f8-8467-43db-a869-9af85bab5f16 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.181674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.182418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.182701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.182978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.193005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:46.200889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:46.207926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:46.208887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 631/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:46.209716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.305839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ecee6-8ee4-4f8a-922f-63f67858ab12 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.334798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:46.339359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.340122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.340408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.340680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.341452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:46.342064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:46.343475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:46.352181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 619/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:46.358163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.434545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:46.457181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd98d5-c1f2-40f1-b0d8-542248afee3c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.489558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.490333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.490616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.490889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.506875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:46.508332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:46.525290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:46.528527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:46.529675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:46.529675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:46.529675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:46.529675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:46.529675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 632/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:46.532505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.633012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436c447-ba3f-454c-bae1-582ec549af72 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.669133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.669793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.670075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.670347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.671067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:46.671370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:46.672724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:46.685547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 620/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:46.691959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.790231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36769168-b2f3-46c6-bffb-eb93d7f9f027 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.815470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:46.822906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.823691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.823971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.824258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.832070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:46.834161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:46.847681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:46.848833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:46.850215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 633/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:33:46.854767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:46.910613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:46.954423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2792ee-568b-431e-a31d-ec8369620dc8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:46.989297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:46.989959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:46.990238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:46.990511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:46.991339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:46.991647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:46.993063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.009877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 621/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:47.015680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.114915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f803a14-6fd6-4176-8f86-11fcc889a40f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:47.148221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:47.148861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:47.149200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:47.153988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:47.205383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:47.217719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:47.219172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.316192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 634/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:33:47.322157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.419998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f7da-c252-4ce4-815d-27a40a5d1bd6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:47.433518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:47.453518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:47.454297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:47.454577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:47.454849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:47.464985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:47.466375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:47.484310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:47.491985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:47.493146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:47.493146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:47.493146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:47.493146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:47.493146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 622/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:47.493521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.576392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:47.588472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d6b5b-1ff5-4298-b5dd-5a8031f55915 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:47.622789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:47.623440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:47.623722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:47.623995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:47.624708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:47.625010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:47.626393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.637391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 635/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:47.643672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.743834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb2632-2046-4de3-af9d-6524bb2628e8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:47.777719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:47.778508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:47.778783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:47.779050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:47.786440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:47.788147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:47.801830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:47.802896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:47.804048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 623/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:47.808853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.909197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dded8-4a4e-4855-ae75-20ca7954f833 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:47.941910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:47.942564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:47.942851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:47.943141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:47.943963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:47.944276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:47.945746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761ef64-6a4a-40cd-8671-da587b891d6a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:47.958426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 636/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:47.964509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:47.990993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:48.065714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190032e-d2c9-4366-9413-c65c0371de56 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:48.099111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:48.099759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:48.100043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:48.100320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:48.101171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=c73b6a617d4546599d336eeb5b45e858) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:48.101488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:48.102921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d4029-8fd8-4d2a-b0ff-85d0a163b10a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:48.108244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 624/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:33:47 2022] GET /identity/v3/role_assignments?user.id=c73b6a617d4546599d336eeb5b45e858&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:50.097453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fadc-dec6-48a0-b1e7-68e59a3b4a3f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.098008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fadc-dec6-48a0-b1e7-68e59a3b4a3f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.098008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385fadc-dec6-48a0-b1e7-68e59a3b4a3f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.099402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 637/1261] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:50.104735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.105191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.105303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.122649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:50.123834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:50.192774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 625/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:50.215338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385fadc-dec6-48a0-b1e7-68e59a3b4a3f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:50.226469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:50.318613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385fadc-dec6-48a0-b1e7-68e59a3b4a3f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:50.351638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:50.352349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.352625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.352894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.363547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:50.371072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:50.378744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:50.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 638/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:50.380672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:50.396537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:50.477759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388625f8-ae36-44b8-920e-16d8511e5ed7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:50.510733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:50.511392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.511675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.511955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.512690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:50.512994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:50.514377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:50.522445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 626/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:50.528248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:50.597254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:50.629539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820b4ac-150d-4a86-8e10-f04864467252 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:50.662722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:50.663473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.663748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.664018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.674508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:50.678223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:50.692771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:50.693881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:50.694837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:50.694837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:50.694837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:50.694837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:50.694837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 639/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:50.699735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:50.782548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:50.801701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760218-656d-434d-8001-1d60c5df5b5f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:50.835798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:50.836496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.836775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.837064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:50.837795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:50.838099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:50.839476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:50.851852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 627/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:50.857707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:50.964668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737edea-cf5e-43b5-8a29-8eb35d266feb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:50.983778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:50.996787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:50.997599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:50.997878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:50.998153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.005782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.007216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:51.021785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:51.023032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:51.024123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 640/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:51.028924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.130319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ef523-4cca-460c-b54c-83ef46f12782 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.164545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.165264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.165542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.165814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.166637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:51.166942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.168356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.177249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:51.186340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 628/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:51.192202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.291569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c4a1-1d44-4279-8f9e-aa726a97764f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.326223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.327006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.327283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.327550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.340011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.341534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:51.360261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:51.361967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:51.363205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:51.363205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:51.363205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:51.363205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:51.363205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 641/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:51.368005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.467910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f957bc0-b228-4bb1-800b-93687e2b82f4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.501868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.502652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.502949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.503224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.503939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:51.504243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.505696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.516836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 629/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:51.523366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.557561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:51.625790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38766a8-b138-47ab-ae4a-7c5a379952d0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.658993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.659770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.660053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.660327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.666817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.668246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:51.682765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:51.683810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:51.684859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 642/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:51.690495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.748264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:51.789471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213a10-cbd2-4ebd-ae5d-d26445f50d88 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.821994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.822575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.822853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.823124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.823908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:51.824215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.825672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76c111-3010-4f26-b311-0ec526f0b386 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:51.837855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 630/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:51.843844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:51.936388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:51.945258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1adaa-be6e-4d07-a2a4-9c7a6270e182 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:51.978682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:51.979595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:51.979877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:51.980151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] PATH_INFO: `/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/c73b6a617d4546599d336eeb5b45e858/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:51.982140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab5797a37f470ebec8ac33b02a8e60, user_id=c73b6a617d4546599d336eeb5b45e858, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:51.983594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7fe09-855e-4f20-ad39-0d9417c7f583 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:52.080831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 643/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Sat Jul 16 12:33:51 2022] PUT /identity/v3/projects/c0ab5797a37f470ebec8ac33b02a8e60/users/c73b6a617d4546599d336eeb5b45e858/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:33:54.167393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514069-73f5-418d-867b-7a480f218296 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.167962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514069-73f5-418d-867b-7a480f218296 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.167962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514069-73f5-418d-867b-7a480f218296 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.169358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 631/1274] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:54.174962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.175422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.175512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.193363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:54.194916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:54.263415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 644/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:54.287899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06514069-73f5-418d-867b-7a480f218296 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:54.390302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06514069-73f5-418d-867b-7a480f218296 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:54.408185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:54.425824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:54.427105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.427593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.428079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.440053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 12:33:54.447675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:54.451139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:54.452197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 632/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Sat Jul 16 12:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:33:54.455788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:54.538533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:54.555816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0d6f4-f6a8-4b47-aabe-e24b99186487 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:54.589511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:54.590169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.590450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.590726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.591470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:54.591776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:54.593156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:54.601803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 645/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Sat Jul 16 12:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:54.607705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:54.708064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e42261-cb6a-44e0-92e7-c74e00929c79 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:54.740856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:54.741740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.742023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.742296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.753530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:54.755044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:54.771891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:54.772961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:54.773953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:54.773953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:54.773953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:54.773953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:54.773953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 633/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:54.779198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:54.879134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a29c2b-27eb-434c-b632-82255a596dc9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:54.912069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:54.912717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:54.912996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:54.913296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:54.914004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:54.914310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:54.915651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:54.928213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 646/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:54.934067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:54.964528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:55.032190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1207f-0b72-47b5-aa98-df7f80cb4ef1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:55.065971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:55.066758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:55.067039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:55.067310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:55.075547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:55.076968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 12:33:55.091587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:55.092586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:55.093595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 634/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 12:33:55.099387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:55.103239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:55.201008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e88611-9f46-482c-903a-a5f441f6679c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:55.236475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:55.237077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:55.237348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:55.237622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:55.238452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:55.238759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:55.240177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:55.257239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 647/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Sat Jul 16 12:33:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:55.262742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:55.361842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:55.371801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba9ec9-eff6-483d-99e9-ad3e2058b492 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:55.409078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:55.409339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:55.409592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:55.409861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:55.485934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:55.487619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:55.589986 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:55.600555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 635/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Sat Jul 16 12:33:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 12:33:55.606436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:55.672782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:55.711849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86ef7d-4ee0-4d3a-94a4-e0c8dbcecbb5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:55.746979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:55.747721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:55.748005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:55.748304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:55.759436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:55.760906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 12:33:55.778844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:55.779989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:55.780953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 12:33:55.780953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 12:33:55.780953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 12:33:55.780953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:55.780953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 648/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:55.785900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:55.884013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadecf7-51ec-41ae-8607-9ecee913668c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:55.916402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:55.917077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:55.917362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:55.917638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:55.918358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:55.918662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:55.920014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:55.932043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 636/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Sat Jul 16 12:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:55.937897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:56.034903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2510e4-881c-4b9d-bb37-fb3bb904731c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:56.048066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:56.066861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:56.067630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:56.067908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:56.068179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:56.074747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:56.076316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 12:33:56.091272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:56.092364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 16 12:33:56.094896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 649/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Sat Jul 16 12:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:56.099315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:56.148068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:56.196404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d392d-9206-41a0-b6c9-bbf3a5028009 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:56.231311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:56.231962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:56.232603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:56.232880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:56.233692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:56.233999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:56.235404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a9b7-4b38-443f-a52f-b04f68747bdc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:56.247794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 637/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Sat Jul 16 12:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:56.254764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:56.352161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63e12c-0dca-405d-ba7b-1ea9ce081502 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:56.385327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:56.386001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:56.386284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:56.386560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:56.387378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] RBAC: Adding query filter params (role.id=a495d38f67e447309647ac782785bd87, scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60, user.id=c73b6a617d4546599d336eeb5b45e858) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:56.387690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:56.389184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754d2c-7127-4981-925e-9dbc4b7047d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:56.395589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 650/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Sat Jul 16 12:33:56 2022] GET /identity/v3/role_assignments?user.id=c73b6a617d4546599d336eeb5b45e858&scope.project.id=c0ab5797a37f470ebec8ac33b02a8e60&role.id=a495d38f67e447309647ac782785bd87 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:58.408629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f575-ab9a-4af2-a72d-9d4d0104ae8d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:58.409681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f575-ab9a-4af2-a72d-9d4d0104ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:58.409853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f575-ab9a-4af2-a72d-9d4d0104ae8d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:58.411289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 638/1288] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:33:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:33:58.416617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:58.417105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:58.417294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:58.433792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:33:58.434903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:33:58.464474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:58.502556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 651/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:33:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:33:58.525105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6f575-ab9a-4af2-a72d-9d4d0104ae8d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:58.628198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6f575-ab9a-4af2-a72d-9d4d0104ae8d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:58.639591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:33:58.663480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:58.664252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:58.664528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:58.664796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:58.665470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:58.666816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:33:58.678642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:33:58.679982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:33:58.681179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 639/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:33:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:33:58.690750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:58.782064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72647d-a4cf-4267-9b5e-cdbbc64f1186 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:58.815241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:58.815910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:58.816174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:58.816454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:58.817175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:58.817480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:58.818827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c75f-e5eb-48b0-8ad5-94457c65e6d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:58.823550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 652/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Sat Jul 16 12:33:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:33:58.830024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:33:58.927185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5c35c-51be-4126-94ee-b5c8248dbb48 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:33:58.959983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:33:58.960561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:33:58.960839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:33:58.961144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:33:58.961838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:33:58.962140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:33:58.963480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79183f10-0030-4b1e-a045-4972d364558f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:33:58.967879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 640/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Sat Jul 16 12:33:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:34:00.957817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4485-253d-427b-b5a0-fe14996b4acd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:00.958393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4485-253d-427b-b5a0-fe14996b4acd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:00.958393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea4485-253d-427b-b5a0-fe14996b4acd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:00.959795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 653/1293] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:34:00.964941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a936b-2dc1-466a-97d7-5fd5c40e01ac None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:00.965396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a936b-2dc1-466a-97d7-5fd5c40e01ac None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:00.965510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a936b-2dc1-466a-97d7-5fd5c40e01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:00.982183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-204a936b-2dc1-466a-97d7-5fd5c40e01ac None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:34:00.983461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-204a936b-2dc1-466a-97d7-5fd5c40e01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:34:01.012689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:01.049500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 641/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:34:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:34:01.072160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea4485-253d-427b-b5a0-fe14996b4acd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:01.106118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:01.175211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea4485-253d-427b-b5a0-fe14996b4acd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:01.211904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:01.212496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:01.212775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:01.213068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:01.213797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:01.215149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2772c-0de2-4847-849a-22275256e83e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:01.275163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 654/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Sat Jul 16 12:34:01 2022] POST /identity/v3/services => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:34:03.315557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09061-e0f7-44a5-80f9-603eca935169 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:03.316075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09061-e0f7-44a5-80f9-603eca935169 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:03.316191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09061-e0f7-44a5-80f9-603eca935169 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:03.317529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 642/1296] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:34:03.322733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:03.323102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:03.323296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:03.340771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:34:03.341993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:34:03.420805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 655/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:34:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:34:03.443666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef09061-e0f7-44a5-80f9-603eca935169 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:03.544842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef09061-e0f7-44a5-80f9-603eca935169 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:03.577954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:03.578739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:03.579594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:03.580035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:03.580428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:03.582147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:34:03.593649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:34:03.595469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:03.596357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 643/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:34:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:34:03.601605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:03.619614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:03.697564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b346cb-1b1b-429b-991a-3e7b9cad73f7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:03.731624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:03.732270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:03.732549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:03.732819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:03.733548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:34:03.733851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:03.735250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:03.741175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 656/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Sat Jul 16 12:34:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:34:03.747188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:03.779389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:03.846778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33b97d-ed53-4eb2-8145-c04132fe6c00 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:03.882272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:03.882995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:03.883274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:03.883553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:03.884348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] RBAC: Adding query filter params (interface=public, service_id=bfa8ea215d80402da0a720c70084b494, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:34:03.884661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:03.886058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b44ba4-9fa5-4bb5-9957-2c4111d7b97e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:03.891076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 644/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Sat Jul 16 12:34:03 2022] GET /identity/v3/endpoints?service_id=bfa8ea215d80402da0a720c70084b494&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:34:03.897919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:03.979150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:03.995917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276002f1-fa88-4c4c-aa4e-6d680e9621f0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:04.028603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:04.029278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:04.029559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:04.029832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:04.030500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:04.031851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848a905-c038-4d0c-8736-b758395a2c22 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:04.039582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 657/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Sat Jul 16 12:34:03 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:34:06.011209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c81c-9792-4b62-8ef6-ce4c47065bb3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:06.011744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c81c-9792-4b62-8ef6-ce4c47065bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:06.011744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4c81c-9792-4b62-8ef6-ce4c47065bb3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:06.013115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 645/1302] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:34:06.018397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:06.018844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:06.018936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:06.035907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:34:06.037095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:34:06.103839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 658/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:34:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 12:34:06.126689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c81c-9792-4b62-8ef6-ce4c47065bb3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:06.222084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:06.229026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4c81c-9792-4b62-8ef6-ce4c47065bb3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:06.263220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:06.264054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:06.264336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:06.264612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:06.265289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:06.266687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 12:34:06.278394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 12:34:06.282489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 16 12:34:06.283686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 646/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Sat Jul 16 12:34:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 12:34:06.286592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:06.361160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:06.383020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d649f4-f314-4c84-a4ff-f16aff1e6b90 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:06.416748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:06.417448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:06.417732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:06.418006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:06.418711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:34:06.419019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:06.420426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa2c7-42a5-432b-b1cf-96f24ce2cb58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:06.425856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 659/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Sat Jul 16 12:34:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:34:06.431847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:34:06.531235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f87f5-6b04-4354-b686-9b3fcd7525bb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:34:06.563188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:34:06.564338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:34:06.564338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:34:06.564338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:34:06.565089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:34:06.566639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c66f0c-2421-4f30-87ca-0a2890be5318 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:34:06.680729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 647/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Sat Jul 16 12:34:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:34:16.422511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=30 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:34:16.675933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:01.258891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432c2d-58f3-45b6-826f-cd6b3494e08b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:01.258891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432c2d-58f3-45b6-826f-cd6b3494e08b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:01.258891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0432c2d-58f3-45b6-826f-cd6b3494e08b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:01.259654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 660/1307] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:01.265848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbc5f-ac05-46da-9745-7d08437fd17f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:01.266309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbc5f-ac05-46da-9745-7d08437fd17f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:01.266394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbbc5f-ac05-46da-9745-7d08437fd17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:01.438824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-10fbbc5f-ac05-46da-9745-7d08437fd17f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:01.440040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-10fbbc5f-ac05-46da-9745-7d08437fd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:01.649441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 648/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:01.711295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:01.711574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:01.712066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:01.713964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 661/1309] 10.222.0.47 () {54 vars in 964 bytes} [Sat Jul 16 12:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:01.733956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4556-f0d6-4fb6-8483-0cc15cad0233 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:01.734277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4556-f0d6-4fb6-8483-0cc15cad0233 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:01.734522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4556-f0d6-4fb6-8483-0cc15cad0233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:01.868246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea5a4556-f0d6-4fb6-8483-0cc15cad0233 None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:01.869637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea5a4556-f0d6-4fb6-8483-0cc15cad0233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:01.897634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:01.990079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 649/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Sat Jul 16 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:01.990695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:01.992436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:02.120911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c513f8d-5e8f-43b2-99f4-9891e871915d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:02.395861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:02.395861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:02.395861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:02.395861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:02.396884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:02.396884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:02.396884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:02.396884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42323646-0648-4d7c-b03f-cf64a5e9f164 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:02.396884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 662/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:04.734286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa287-06a5-4999-8944-66e175dcc9c2 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:04.734852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa287-06a5-4999-8944-66e175dcc9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:04.734852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa287-06a5-4999-8944-66e175dcc9c2 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:04.736224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 650/1312] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:04.741619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51402e-2314-4a85-999d-a5b47d9b15a4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:04.742033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51402e-2314-4a85-999d-a5b47d9b15a4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:04.742171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51402e-2314-4a85-999d-a5b47d9b15a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:04.813439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-de51402e-2314-4a85-999d-a5b47d9b15a4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:04.814685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-de51402e-2314-4a85-999d-a5b47d9b15a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:04.919914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 663/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:40:04.975205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:04.975523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:04.975807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:04.977280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 651/1314] 10.222.0.47 () {54 vars in 964 bytes} [Sat Jul 16 12:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:04.983610 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b5f4a-ba29-401d-bcde-70954fc93be3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:04.983901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b5f4a-ba29-401d-bcde-70954fc93be3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:04.984176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b5f4a-ba29-401d-bcde-70954fc93be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:04.994485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:05.057354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-508b5f4a-ba29-401d-bcde-70954fc93be3 None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:05.058546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-508b5f4a-ba29-401d-bcde-70954fc93be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:05.134252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 664/1315] 10.222.0.47 () {58 vars in 1061 bytes} [Sat Jul 16 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:05.144086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:05.145672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:05.256957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f767f31-6b57-46cf-aef1-09b60b69ac32 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:05.327944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:05.330604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:05.331245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:05.331525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:05.331797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:05.333463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:05.440194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:05.441657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901a372-f44d-48ab-b64b-0dc9b99b8e2e service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:05.460308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 652/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:08.180703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:08.181138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:08.181300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:08.182685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 665/1317] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:08.188100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2fb7-3306-40ba-890c-79ca2ed5d37f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:08.188500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2fb7-3306-40ba-890c-79ca2ed5d37f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:08.188656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef2fb7-3306-40ba-890c-79ca2ed5d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:08.253298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-81ef2fb7-3306-40ba-890c-79ca2ed5d37f None None] MFA Rules not processed for user `f3b3c2d0f7ab4159b27ebbffbf123f03`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:08.254486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-81ef2fb7-3306-40ba-890c-79ca2ed5d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:08.304553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:08.349291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 653/1318] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:08.389103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:08.390632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:08.493483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf888b-62de-4809-94e5-c83a617d70da None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:08.496336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:08.568177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:08.568818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:08.569129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:08.569407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:08.571053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:08.677363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:08.678793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8be4f7-8e93-4c31-8f83-d399a8382acb service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:08.696798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 666/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:11.495482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:11.495998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:11.495998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:11.497459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 654/1320] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:11.502558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313a7fb-2cf3-4118-890a-1d4d65d1dd87 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:11.502954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313a7fb-2cf3-4118-890a-1d4d65d1dd87 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:11.503114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313a7fb-2cf3-4118-890a-1d4d65d1dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:11.541311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:11.565986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6313a7fb-2cf3-4118-890a-1d4d65d1dd87 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:11.567171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6313a7fb-2cf3-4118-890a-1d4d65d1dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:11.666714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 667/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:11.705461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:11.706990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:11.809113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0465da6-4df1-404e-ac6a-c5c9a2f7e55d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:11.880521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:11.881202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:11.881483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:11.881756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:11.883372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:11.891392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:11.990959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:11.992383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61782440-eec6-49f4-ba57-b28ca950223e service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:12.010558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 655/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:14.997157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:14.997699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:14.997699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:14.999093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 668/1323] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:15.004318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2da78-db4d-4e26-adee-b9e4cb073df5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:15.004745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2da78-db4d-4e26-adee-b9e4cb073df5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:15.005003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2da78-db4d-4e26-adee-b9e4cb073df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:15.069653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-84a2da78-db4d-4e26-adee-b9e4cb073df5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:15.070840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-84a2da78-db4d-4e26-adee-b9e4cb073df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:15.128501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:15.167068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 656/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:15.204949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:15.206514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:15.310224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb075ae-d407-4c80-b65a-9010becc9dc4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:15.323570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:15.390600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:15.391263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:15.391525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:15.391793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:15.393525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:15.507675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:15.509185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8405-506d-4ce7-8ee7-229301acccc9 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:15.529146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 669/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:17.615188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:17.615720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:17.615720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:17.617132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 657/1326] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:17.622796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef77ba-5c3a-458b-a48e-fdd31c1d798d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:17.623233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef77ba-5c3a-458b-a48e-fdd31c1d798d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:17.623410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef77ba-5c3a-458b-a48e-fdd31c1d798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:17.670385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:17.689307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-34ef77ba-5c3a-458b-a48e-fdd31c1d798d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:17.690522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-34ef77ba-5c3a-458b-a48e-fdd31c1d798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:17.786508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 670/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:17.821997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:17.823536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:17.928090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432aed5-d276-4e78-b12e-5cbb8cb5a4e9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:18.003509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:18.004178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:18.004443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:18.004716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:18.006482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:18.023771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:18.115549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:18.116985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ecf3-9856-4d59-9eef-5a3666b0ea1b service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:18.135072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 658/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:20.251738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:20.252210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:20.252306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:20.253719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 671/1329] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:20.259192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0f48-27ba-4748-a0a8-5138e19a80c6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:20.259652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0f48-27ba-4748-a0a8-5138e19a80c6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:20.259772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0f48-27ba-4748-a0a8-5138e19a80c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:20.329064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-701a0f48-27ba-4748-a0a8-5138e19a80c6 None None] MFA Rules not processed for user `f3b3c2d0f7ab4159b27ebbffbf123f03`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:20.330333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-701a0f48-27ba-4748-a0a8-5138e19a80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:20.397239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:20.429609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 659/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:20.468761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:20.470323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:20.575877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0841952-c917-43cd-ae49-182f01ed49e9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:20.597263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:20.655529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:20.656173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:20.656456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:20.656728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:20.658381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:20.768871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:20.770253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794812b-7136-4a81-acd7-b94ebe15f2cd service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:20.789171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 672/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:40:23.283436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:23.283950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:23.284062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:23.285378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 660/1332] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:23.290881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34780-b142-4a49-b4a2-95f732121e24 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:23.291299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34780-b142-4a49-b4a2-95f732121e24 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:23.291416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34780-b142-4a49-b4a2-95f732121e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:23.355251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:23.358065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-66c34780-b142-4a49-b4a2-95f732121e24 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:23.359557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-66c34780-b142-4a49-b4a2-95f732121e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:23.460177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 673/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:23.496632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:23.498186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:23.597662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ccd2f-3332-4506-804f-f38e32f9b953 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:23.670421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:23.671065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:23.671336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:23.671603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:23.673255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:23.700604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:23.785138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:23.786506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e338fd5-cee7-470d-b537-462ee375b820 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:23.804889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 661/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:25.859220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:25.859824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:25.859824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:25.861173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 674/1335] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:25.866220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d897-d05e-4f11-a901-56d9e7c88ef0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:25.866623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d897-d05e-4f11-a901-56d9e7c88ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:25.866773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d897-d05e-4f11-a901-56d9e7c88ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:25.929965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1c14d897-d05e-4f11-a901-56d9e7c88ef0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:25.931167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1c14d897-d05e-4f11-a901-56d9e7c88ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:26.003973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:26.025928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 662/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:26.067824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:26.069380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:26.169703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790c2ee-2212-45df-b957-acd7432a2b4e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:26.198653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:26.244316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:26.244959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:26.245266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:26.245537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:26.247488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:26.352796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:26.354275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abf502-26e2-4aa7-99ef-d5312d936e9e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:26.373334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 675/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:28.921383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:28.921940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:28.921940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:28.923319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 663/1338] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:28.928533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715ba7-7be7-4ced-af89-919c8059f1d5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:28.931051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715ba7-7be7-4ced-af89-919c8059f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:28.931051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715ba7-7be7-4ced-af89-919c8059f1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:28.993898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fb715ba7-7be7-4ced-af89-919c8059f1d5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:28.995127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb715ba7-7be7-4ced-af89-919c8059f1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:29.003341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:29.093107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 676/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:29.128329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:29.129881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:29.232948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80831291-9742-42f5-85a1-3bcb4ce0c107 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:29.305004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:29.305676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:29.305959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:29.306233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:29.307855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:29.342525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:29.413715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:29.415143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87980e-b3bc-4810-8cea-273b14314496 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:29.434782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 664/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:33.622615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:33.622615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:33.622615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:33.622615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 677/1341] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:33.624147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214761fc-93e2-4461-bf02-d6eeec431761 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:33.624523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214761fc-93e2-4461-bf02-d6eeec431761 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:33.624697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214761fc-93e2-4461-bf02-d6eeec431761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:33.692148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-214761fc-93e2-4461-bf02-d6eeec431761 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:33.693388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-214761fc-93e2-4461-bf02-d6eeec431761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:33.790642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:33.802046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 665/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:33.843166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:33.844700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:33.947558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9ee02-2c38-452d-a54b-01b619c9a7ad None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:33.986281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:34.023071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:34.023728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:34.024006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:34.024293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:34.025980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:34.136693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:34.138187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b8ff3-3734-4d6e-8f52-80a3aeffb887 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:34.157258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 678/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:37.488411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:37.488985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:37.488985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:37.490396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 666/1344] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:37.495634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bfb5-2ddb-423e-bfac-5d0b37650dc4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:37.496085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bfb5-2ddb-423e-bfac-5d0b37650dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:37.496201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bfb5-2ddb-423e-bfac-5d0b37650dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:37.560613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-be74bfb5-2ddb-423e-bfac-5d0b37650dc4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:37.561862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-be74bfb5-2ddb-423e-bfac-5d0b37650dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:37.579912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:37.660874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 679/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:37.700224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:37.702167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:37.802891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa2e42-c886-4861-8f02-2264c090744a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:37.878600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:37.879254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:37.879540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:37.879816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:37.881477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:37.929163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:37.999522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:38.000963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d06772-f578-414f-9499-a228ca2172cc service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:38.020578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 667/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:42.387470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:42.387962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:42.388053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:42.389438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 680/1347] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:42.394963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef44ea-fc0b-422e-ba12-bdb98f0a0799 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:42.395431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef44ea-fc0b-422e-ba12-bdb98f0a0799 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:42.395520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef44ea-fc0b-422e-ba12-bdb98f0a0799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:42.467476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6aef44ea-fc0b-422e-ba12-bdb98f0a0799 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:42.468711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6aef44ea-fc0b-422e-ba12-bdb98f0a0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:42.572485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:42.575390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 668/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:42.615459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:42.616977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:42.725550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63635e-6b7c-46a6-a9cd-9500b3258331 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:42.773513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:42.799276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:42.800155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:42.800626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:42.801224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:42.802850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:42.907051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:42.908471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860b42a-59a1-43f5-baf3-eb798e1b58b0 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:42.927056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 681/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:40:47.791845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be8fb-d380-43ca-aae8-ecfe893abe0e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:47.792443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be8fb-d380-43ca-aae8-ecfe893abe0e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:47.792443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be8fb-d380-43ca-aae8-ecfe893abe0e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:47.794619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 669/1350] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:47.800001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac6e3-691c-4c10-98ca-31abd9b9884d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:47.800488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac6e3-691c-4c10-98ca-31abd9b9884d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:47.800598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac6e3-691c-4c10-98ca-31abd9b9884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:47.964423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2e1ac6e3-691c-4c10-98ca-31abd9b9884d None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:47.966164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2e1ac6e3-691c-4c10-98ca-31abd9b9884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:47.988890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:48.090378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 682/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:52.230297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4451-bb31-48f1-a0b3-dc96765bce53 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:52.231147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4451-bb31-48f1-a0b3-dc96765bce53 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:52.231578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4451-bb31-48f1-a0b3-dc96765bce53 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:52.233142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 670/1352] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:52.238607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a1e2-3fbd-42bd-8407-417b543bac17 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:52.239032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a1e2-3fbd-42bd-8407-417b543bac17 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:52.239222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a1e2-3fbd-42bd-8407-417b543bac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:52.318860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4422a1e2-3fbd-42bd-8407-417b543bac17 None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:52.320072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4422a1e2-3fbd-42bd-8407-417b543bac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:52.395833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:52.434110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 683/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:40:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:53.763243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392d50-86e7-46a8-bbc6-53b8471b51e0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:53.763840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392d50-86e7-46a8-bbc6-53b8471b51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:53.764248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392d50-86e7-46a8-bbc6-53b8471b51e0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:53.765671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 671/1354] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:53.771883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b467-95f7-47a9-a576-94e1ff0b0f33 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:53.772793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b467-95f7-47a9-a576-94e1ff0b0f33 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:53.773491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b467-95f7-47a9-a576-94e1ff0b0f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:53.848953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-42b5b467-95f7-47a9-a576-94e1ff0b0f33 None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:53.850229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-42b5b467-95f7-47a9-a576-94e1ff0b0f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:53.961159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 684/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:40:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:57.413956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004e87-7cf9-4eb9-b5c3-468aea701150 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:57.414833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004e87-7cf9-4eb9-b5c3-468aea701150 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:57.415112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004e87-7cf9-4eb9-b5c3-468aea701150 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:57.419951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 672/1356] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:40:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:57.423465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91a77f-aa08-461c-95d4-81957f135d3a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:57.423923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91a77f-aa08-461c-95d4-81957f135d3a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:57.424150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91a77f-aa08-461c-95d4-81957f135d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:57.470705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:57.503842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1e91a77f-aa08-461c-95d4-81957f135d3a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:57.505032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e91a77f-aa08-461c-95d4-81957f135d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:57.607375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 685/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:40:57.661030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:57.661657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:57.662317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:57.664682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 673/1358] 10.222.0.47 () {54 vars in 961 bytes} [Sat Jul 16 12:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:40:57.672517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440b341-0bdb-4e5c-9bb3-3bd455039c91 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:57.679208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440b341-0bdb-4e5c-9bb3-3bd455039c91 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:57.679208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440b341-0bdb-4e5c-9bb3-3bd455039c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:57.764461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8440b341-0bdb-4e5c-9bb3-3bd455039c91 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:40:57.765623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8440b341-0bdb-4e5c-9bb3-3bd455039c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:57.816798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:57.901648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 686/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Sat Jul 16 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:40:57.912825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:40:57.914437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:58.069666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b5691-2d8c-40cc-8cc2-78e0a7f96b6b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:40:58.157809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:40:58.200758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:40:58.201690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:40:58.201974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:40:58.202245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:40:58.203869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:40:58.297696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:40:58.299324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224534f5-e156-4510-bd6f-bed4ca36c09e service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:40:58.318036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 674/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:00.544879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:00.545467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:00.545467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:00.547051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 687/1361] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:41:00.553055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115a147-b9f1-4a43-ba6d-60973fae7203 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:00.553981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115a147-b9f1-4a43-ba6d-60973fae7203 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:00.555134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115a147-b9f1-4a43-ba6d-60973fae7203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:00.642022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d115a147-b9f1-4a43-ba6d-60973fae7203 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:00.643831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d115a147-b9f1-4a43-ba6d-60973fae7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:00.695870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:00.768924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 675/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:00.818156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:00.820449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:01.009878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4dc24-8b6b-43c7-a269-f57cb39fae29 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:01.021226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:01.158082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:01.158998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:01.159458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:01.159925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:01.161815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:01.266514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:01.268120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804110f-7a65-485c-9a6b-47571ce41a8a service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:01.289199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 688/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:03.787067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b833aec-63f7-4612-9d72-9a649f67601c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:03.787786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b833aec-63f7-4612-9d72-9a649f67601c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:03.787786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b833aec-63f7-4612-9d72-9a649f67601c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:03.789228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 676/1364] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:03.794774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aeb43-b064-4230-9d74-423618cd1fb6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:03.795214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aeb43-b064-4230-9d74-423618cd1fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:03.795390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aeb43-b064-4230-9d74-423618cd1fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:03.811562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:03.869621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9d2aeb43-b064-4230-9d74-423618cd1fb6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:03.870998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9d2aeb43-b064-4230-9d74-423618cd1fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:03.977109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 689/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:41:04.024691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:04.024983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:04.025293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:04.027040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 677/1366] 10.222.0.47 () {54 vars in 961 bytes} [Sat Jul 16 12:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:04.033093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1505f-3bba-4eb6-8256-e98c10feeef0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:04.033939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1505f-3bba-4eb6-8256-e98c10feeef0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:04.033939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1505f-3bba-4eb6-8256-e98c10feeef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:04.118024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e6f1505f-3bba-4eb6-8256-e98c10feeef0 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:04.119765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e6f1505f-3bba-4eb6-8256-e98c10feeef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:04.155470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:04.258380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 690/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Sat Jul 16 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:04.273368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:04.275370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:04.429421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab577645-1180-424d-9a50-f1c70b7860fc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:04.450763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:04.548550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:04.549231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:04.549517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:04.549787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:04.551471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:04.645643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:04.647351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b970-a145-4b1d-986d-d2b912785c11 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:04.667807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 678/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:41:05.642041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e62dd-944d-4769-b672-851daa1c2a9c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:05.642615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e62dd-944d-4769-b672-851daa1c2a9c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:05.642615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e62dd-944d-4769-b672-851daa1c2a9c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:05.644159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 691/1369] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:05.649112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67675c02-c667-4607-874e-38e002f4d413 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:05.649538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67675c02-c667-4607-874e-38e002f4d413 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:05.649741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67675c02-c667-4607-874e-38e002f4d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:05.682093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:05.720513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-67675c02-c667-4607-874e-38e002f4d413 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:05.721953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-67675c02-c667-4607-874e-38e002f4d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:05.824529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 679/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:06.192769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:06.200630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:06.200630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:06.200630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 692/1371] 10.222.0.47 () {54 vars in 963 bytes} [Sat Jul 16 12:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:06.201192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244530e-e64d-4e65-adcf-8ea4a8f06196 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:06.201192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244530e-e64d-4e65-adcf-8ea4a8f06196 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:06.201192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244530e-e64d-4e65-adcf-8ea4a8f06196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:06.281254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d244530e-e64d-4e65-adcf-8ea4a8f06196 None None] MFA Rules not processed for user `d10255724102414ca184fac122fd1e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:06.282509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d244530e-e64d-4e65-adcf-8ea4a8f06196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:06.330963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:06.362007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 680/1372] 10.222.0.47 () {58 vars in 1060 bytes} [Sat Jul 16 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:06.369605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:06.371132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:06.482180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecd859-44b2-4baf-8bcd-66bdbcacb624 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:06.524339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:06.559947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:06.560601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:06.560887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:06.561184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:06.562819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:06.677144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:06.678861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabbb16-35a1-41fc-8e0e-443f2f2adad6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:06.701655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 693/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:06.993264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ba832-b183-4942-85a5-292368b68196 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:06.993767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ba832-b183-4942-85a5-292368b68196 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:06.994636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ba832-b183-4942-85a5-292368b68196 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:06.998601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 681/1374] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:07.002764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131361a1-6238-47f3-b59b-657bc31edc06 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:07.003115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131361a1-6238-47f3-b59b-657bc31edc06 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:07.003351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131361a1-6238-47f3-b59b-657bc31edc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:07.070957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-131361a1-6238-47f3-b59b-657bc31edc06 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:07.073065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-131361a1-6238-47f3-b59b-657bc31edc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:07.097654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:07.180803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 694/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:07.232213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ba832-b183-4942-85a5-292368b68196 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:07.233774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-948ba832-b183-4942-85a5-292368b68196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:07.380158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ba832-b183-4942-85a5-292368b68196 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:07.431454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:07.495030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:07.495679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:07.495961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:07.496241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:07.497933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:07.586301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:07.588530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704617f-8702-4aa9-9852-6d2a43a1f12b service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:07.608074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 682/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:09.797746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:09.798236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:09.798330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:09.799830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 695/1377] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:09.804870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f94953-30f3-43eb-9d48-9b5069b24e65 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:09.805347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f94953-30f3-43eb-9d48-9b5069b24e65 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:09.805456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f94953-30f3-43eb-9d48-9b5069b24e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:09.887413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-29f94953-30f3-43eb-9d48-9b5069b24e65 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:09.888558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-29f94953-30f3-43eb-9d48-9b5069b24e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:09.895818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:09.987274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 683/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:10.032303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:10.033875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:10.191708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dadc9-e450-40e7-9dc8-e34eec2a580d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:10.195741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:10.340459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:10.341729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:10.342126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:10.342539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:10.344476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:10.456534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:10.458283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6f24-7655-4124-8a35-534ce9a21d93 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:10.478956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 696/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:12.681093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:12.681093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:12.681093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:12.688459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 684/1380] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:12.688637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0efc-7784-4bce-815f-671bf37866f0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:12.688637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0efc-7784-4bce-815f-671bf37866f0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:12.688637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0efc-7784-4bce-815f-671bf37866f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:12.702850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:12.761940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c3df0efc-7784-4bce-815f-671bf37866f0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:12.763151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c3df0efc-7784-4bce-815f-671bf37866f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:12.858520 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 697/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:41:12.902591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:12.904096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:13.049556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:13.060143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9785a-4fec-4977-80b0-dceb687cc89f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:13.186635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:13.187289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:13.187574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:13.188005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:13.189717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:13.296053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:13.297847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707775f8-e4ab-4ea5-bbd2-391fc915e772 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:13.309669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:13.319184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 685/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:15.722916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:15.723471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:15.723657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:15.725540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 698/1383] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:15.730840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083c30-93dc-4a8d-a1b1-974f0a678f58 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:15.731261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083c30-93dc-4a8d-a1b1-974f0a678f58 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:15.731412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93083c30-93dc-4a8d-a1b1-974f0a678f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:15.799084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-93083c30-93dc-4a8d-a1b1-974f0a678f58 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:15.800318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-93083c30-93dc-4a8d-a1b1-974f0a678f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:15.899019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 686/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:15.943766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:15.945344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:16.032489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:16.105262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69c3a5-b598-4a7c-9416-577ecb0dc89e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:16.236896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:16.237580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:16.237866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:16.238143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:16.239801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:16.291481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:16.331736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:16.333571 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2055c-24eb-44f8-8de4-a3925391853c service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:16.352090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 699/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:18.475819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:18.476351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:18.476351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:18.477779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 687/1386] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:18.484790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8adf-3760-4fb5-a15f-769837e91e26 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:18.485299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8adf-3760-4fb5-a15f-769837e91e26 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:18.485405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8adf-3760-4fb5-a15f-769837e91e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:18.552601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-133e8adf-3760-4fb5-a15f-769837e91e26 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:18.553846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-133e8adf-3760-4fb5-a15f-769837e91e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:18.652936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 700/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:18.702263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:18.704314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:18.803187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:18.873636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6e8a8-6fa2-49f2-83e3-9e5211d533cb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:18.991599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:18.992502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:18.992874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:18.993755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:18.996147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:19.047256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:19.108611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:19.110244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bb53e-86f9-422b-bfed-9bf47ae6efc5 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:19.128381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 688/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:21.353291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:21.353883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:21.353883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:21.355720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 701/1389] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:21.360921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866eb72-97ad-4925-8752-cd3505b424ea None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:21.361391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866eb72-97ad-4925-8752-cd3505b424ea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:21.361509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866eb72-97ad-4925-8752-cd3505b424ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:21.426439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d866eb72-97ad-4925-8752-cd3505b424ea None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:21.427633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d866eb72-97ad-4925-8752-cd3505b424ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:21.522919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 689/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:41:21.567300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:21.568814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:21.601050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:21.720183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85186f-8517-47a3-b650-40a02f3c48c0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:21.825055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:21.836644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:21.837323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:21.837608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:21.837885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:21.839694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:21.929151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:21.930743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9063fc-a14b-436d-bcdc-c55680fcc989 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:21.950177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 702/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:22.096642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede2a1-bb9e-4a8a-970e-4a4e02288513 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:22.096912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede2a1-bb9e-4a8a-970e-4a4e02288513 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:22.097226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede2a1-bb9e-4a8a-970e-4a4e02288513 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:22.098679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 690/1392] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:22.103907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3485e6-7f6d-4d67-ab3b-6edb36da89e3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:22.104704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3485e6-7f6d-4d67-ab3b-6edb36da89e3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:22.104985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3485e6-7f6d-4d67-ab3b-6edb36da89e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:22.179554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4b3485e6-7f6d-4d67-ab3b-6edb36da89e3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:22.180975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4b3485e6-7f6d-4d67-ab3b-6edb36da89e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:22.246805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:22.288552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 703/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:41:22.346042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:22.346328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:22.346601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:22.348026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 691/1394] 10.222.0.47 () {54 vars in 960 bytes} [Sat Jul 16 12:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:22.353006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508c4f-8747-40dd-b2fa-fe2b2b74e6d7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:22.353407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508c4f-8747-40dd-b2fa-fe2b2b74e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:22.353924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508c4f-8747-40dd-b2fa-fe2b2b74e6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:22.429206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0b508c4f-8747-40dd-b2fa-fe2b2b74e6d7 None None] MFA Rules not processed for user `eff6db7c70f64a649a64313389f7e9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:22.430415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0b508c4f-8747-40dd-b2fa-fe2b2b74e6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:22.505477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 704/1395] 10.222.0.47 () {58 vars in 1057 bytes} [Sat Jul 16 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:22.512984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:22.514556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:22.528872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:22.619438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aa96a-1fd2-4849-9477-8c43ae7259aa None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:22.691082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:22.691732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:22.692011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:22.692562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:22.694220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:22.761676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:22.803285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:22.804718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05992ee-675f-4402-b79b-56e9dae19b60 service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:22.824772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 692/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:23.717640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e8c5-8adc-4823-a468-577944ac9d40 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:23.718148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e8c5-8adc-4823-a468-577944ac9d40 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:23.718286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e8c5-8adc-4823-a468-577944ac9d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:23.789836 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a138e8c5-8adc-4823-a468-577944ac9d40 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:23.791037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a138e8c5-8adc-4823-a468-577944ac9d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:23.799845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:23.877431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 705/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:23.925242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:23.925738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:23.925846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:23.927176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 693/1398] 10.222.0.47 () {54 vars in 942 bytes} [Sat Jul 16 12:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:23.934102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b08d-a3ff-4e4d-a7ff-057a64ce9df8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:23.939222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b08d-a3ff-4e4d-a7ff-057a64ce9df8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:23.939222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b08d-a3ff-4e4d-a7ff-057a64ce9df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:24.011105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9288b08d-a3ff-4e4d-a7ff-057a64ce9df8 None None] MFA Rules not processed for user `6d7e37eb8b3e44d5a7f2be9af4a02c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:24.012148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9288b08d-a3ff-4e4d-a7ff-057a64ce9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.099580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:24.121915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 706/1399] 10.222.0.47 () {58 vars in 1039 bytes} [Sat Jul 16 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:24.131243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:24.132769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.141887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022b66-f057-40c8-a1da-6ac82480cfb3 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:24.142179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022b66-f057-40c8-a1da-6ac82480cfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:24.142476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022b66-f057-40c8-a1da-6ac82480cfb3 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:24.143826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 707/1400] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:41:24.149119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:24.153180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:24.153180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:24.220687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:24.221936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.279000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f62cb-2079-4faa-9ba4-2fcadb19d01f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:24.315777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:24.318637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 708/1401] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:41:24.365154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:24.366361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.387702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:24.388355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:24.388639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:24.388912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:24.390875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.431404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:24.475069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:24.476659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55eb89-af3f-4a74-a6c5-14223d7848d0 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:24.483498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79201538-e321-4460-b37f-f9f5e76976fe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:24.485771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 694/1402] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:41:24.601833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:24.602396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:24.602676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:24.602946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:24.605721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:24.664082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:24.695204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:24.696822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bb5b-70dd-4767-b891-4eaa38f3cd98 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:24.716311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 709/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:41:27.019311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:27.019855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:27.019855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:27.021303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 695/1404] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:27.027119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66bfd-4838-4e10-8e2e-5b7dd40114f5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:27.027569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66bfd-4838-4e10-8e2e-5b7dd40114f5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:27.027763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66bfd-4838-4e10-8e2e-5b7dd40114f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:27.099360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-afe66bfd-4838-4e10-8e2e-5b7dd40114f5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:27.100688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-afe66bfd-4838-4e10-8e2e-5b7dd40114f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:27.229470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 710/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:27.265876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:27.267263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:27.343398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:27.424438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5f370-350c-4597-8c8e-e2b9d83ce976 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:27.547754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:27.548315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:27.548600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:27.548876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:27.550569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:27.582318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:27.645046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:27.647020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ce8d6-5996-458d-8bb5-7c028ddc8ffc service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:27.667405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 696/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:29.827692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:29.828227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:29.828227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:29.835382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c251-a95d-4edf-b2e6-17718abe3285 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:29.835940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c251-a95d-4edf-b2e6-17718abe3285 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:29.835940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c251-a95d-4edf-b2e6-17718abe3285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:29.839429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 711/1407] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:41:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:29.909156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fd87c251-a95d-4edf-b2e6-17718abe3285 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:29.910405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fd87c251-a95d-4edf-b2e6-17718abe3285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:30.013356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 697/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:30.057963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:30.061829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:30.100903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:30.222896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9711e-8931-4ffd-89e4-6d02f4695be7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:30.337426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:30.343008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:30.343661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:30.343952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:30.344229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:30.345915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:30.452152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:30.453805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de7c64-129c-4a82-9d1e-e21cfbe6b744 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:30.475603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 712/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:40.011605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:40.472972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=47 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:47.809815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8d7ca-3c78-4653-aed3-3ea69451701c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:47.810395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8d7ca-3c78-4653-aed3-3ea69451701c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:47.810395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8d7ca-3c78-4653-aed3-3ea69451701c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:47.811745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 698/1410] 10.222.0.47 () {54 vars in 918 bytes} [Sat Jul 16 12:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:47.816882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f4b5b-1819-4f2f-96ba-4675c9c0312c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:47.817334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f4b5b-1819-4f2f-96ba-4675c9c0312c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:47.817470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f4b5b-1819-4f2f-96ba-4675c9c0312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:47.889784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-675f4b5b-1819-4f2f-96ba-4675c9c0312c None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:47.890980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-675f4b5b-1819-4f2f-96ba-4675c9c0312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:47.995941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 713/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Sat Jul 16 12:41:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:48.010882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef230e4-8a29-4246-acd9-75e88262af41 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:48.011168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef230e4-8a29-4246-acd9-75e88262af41 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:48.011447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef230e4-8a29-4246-acd9-75e88262af41 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:48.012823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 699/1412] 10.222.0.47 () {54 vars in 918 bytes} [Sat Jul 16 12:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:48.017725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08daa-18e2-4fa5-a6f2-8143c2188b2d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:48.018014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08daa-18e2-4fa5-a6f2-8143c2188b2d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:48.018294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08daa-18e2-4fa5-a6f2-8143c2188b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:48.089166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:48.091691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-26e08daa-18e2-4fa5-a6f2-8143c2188b2d None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:48.092887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26e08daa-18e2-4fa5-a6f2-8143c2188b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:48.208880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 714/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Sat Jul 16 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:50.722991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:50.723580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:50.723580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:50.724941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:50.731074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f5dce-2819-4d54-bc7a-64e37b3eeb7a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:50.731504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f5dce-2819-4d54-bc7a-64e37b3eeb7a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:50.731619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f5dce-2819-4d54-bc7a-64e37b3eeb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:50.795117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-289f5dce-2819-4d54-bc7a-64e37b3eeb7a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:50.796318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-289f5dce-2819-4d54-bc7a-64e37b3eeb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:50.851030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:50.892558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:50.936562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:50.938111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:51.084179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42272422-bf53-4f2a-9e6f-7b9cfe400d7d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:51.155013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:51.195253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:51.195892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:51.196171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:51.196488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:51.198643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:51.285825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:51.287413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a7673-7eab-46c8-858b-7c8b51f400d2 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:51.305165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 701/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:52.212355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:52.212828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:52.212992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:52.214397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:52.219341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f53496-2879-4f2a-a32d-0e4d5b56b1af None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:52.219767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f53496-2879-4f2a-a32d-0e4d5b56b1af None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:52.219899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f53496-2879-4f2a-a32d-0e4d5b56b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:52.293717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-37f53496-2879-4f2a-a32d-0e4d5b56b1af None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:52.294914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-37f53496-2879-4f2a-a32d-0e4d5b56b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:52.322254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:52.389942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:52.542453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:52.543874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:52.688181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e00f-c83c-44e3-aed8-bcd037b1167a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:52.728443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:52.799979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:52.800564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:52.800881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:52.801230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:52.802929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:52.891014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:52.892662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203434da-6e26-43a8-b01f-7b23f32c2f95 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:52.911220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 717/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:53.791855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:53.792423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:53.792423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:53.793822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:53.798885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65497bb-92bb-4975-941d-df26ec155ed8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:53.799316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65497bb-92bb-4975-941d-df26ec155ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:53.799412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65497bb-92bb-4975-941d-df26ec155ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:53.852527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:53.875008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f65497bb-92bb-4975-941d-df26ec155ed8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:53.876209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f65497bb-92bb-4975-941d-df26ec155ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:53.975147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:54.130728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:54.132240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:54.277166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc2ac7-96fa-4c73-afed-2a7eee3b5b7c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:54.289365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:54.397147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:54.397728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:54.398006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:54.398275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:54.401126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:54.504492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:54.506119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b799f96-33b1-412d-8e7d-d9f0e6142c69 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:54.526214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 704/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:55.451455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:55.451989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:55.451989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:55.453410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:55.458304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e6bf-85d0-465e-8d85-95b7d8074607 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:55.458755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e6bf-85d0-465e-8d85-95b7d8074607 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:55.458875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8e6bf-85d0-465e-8d85-95b7d8074607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:55.478460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:55.534517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8de8e6bf-85d0-465e-8d85-95b7d8074607 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:55.535730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8de8e6bf-85d0-465e-8d85-95b7d8074607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:55.630323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:55.784596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:55.786038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:55.895370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:55.930947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2178ca-4b3f-4dfb-8325-98fb7d45dc23 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:56.043307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:56.043878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:56.044163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:56.044438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:56.046118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:56.121295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:56.134659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:56.136253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93d691-d5af-4ece-9bb5-30f01870638c service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:56.154963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 720/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:57.014618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:57.015154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:57.015294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:57.016581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:57.021591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfa84-a1d4-4406-b73d-0c2d79a77a32 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:57.022005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfa84-a1d4-4406-b73d-0c2d79a77a32 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:57.022148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cfa84-a1d4-4406-b73d-0c2d79a77a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:57.097327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e6cfa84-a1d4-4406-b73d-0c2d79a77a32 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:57.098442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e6cfa84-a1d4-4406-b73d-0c2d79a77a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:57.195044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:57.347071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:57.348627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:57.429682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:57.490890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2996e99-6f65-4a80-9fb4-8488255381b7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:57.602373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:57.603540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:57.603823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:57.604094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:57.605763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:57.653439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:57.695336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:57.696922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3167189-0e7c-4b7c-9301-626d8f6c86d0 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:57.715593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 707/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:41:58.589846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:58.590393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:58.590491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:58.592095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:41:58.597095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dae0-5cb2-427d-8b44-fdcc88e4893e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:58.597507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dae0-5cb2-427d-8b44-fdcc88e4893e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:58.597658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dae0-5cb2-427d-8b44-fdcc88e4893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:58.671446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e232dae0-5cb2-427d-8b44-fdcc88e4893e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:41:58.672639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e232dae0-5cb2-427d-8b44-fdcc88e4893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:58.771808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:41:58.931323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:41:58.933099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:58.985208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:59.084506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27162bd5-9646-4ac7-a712-6ab3fbef6a13 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:41:59.206816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:41:59.207464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:41:59.207747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:41:59.208018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:41:59.209680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:41:59.217201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:41:59.299451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:41:59.301087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922323e-483a-43eb-89ae-0d305dbbecd5 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:41:59.318661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 723/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:00.194836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:00.195344 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:00.195463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:00.196945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:00.202291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7426-392a-48e2-9462-1641deb93240 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:00.202700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7426-392a-48e2-9462-1641deb93240 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:00.202847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7426-392a-48e2-9462-1641deb93240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:00.276375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d1aa7426-392a-48e2-9462-1641deb93240 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:00.277605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d1aa7426-392a-48e2-9462-1641deb93240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:00.344332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:00.373182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:00.526350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:00.527801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:00.545996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:00.682016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8072ea-f3eb-457f-9662-3883596672b3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:00.796418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:00.831281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:00.832054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:00.832583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:00.833093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:00.834761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:00.950729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:00.952356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689d338-b5e9-48f5-8c8e-13258b631361 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:00.971868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 710/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:01.853954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:01.854492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:01.854492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:01.855892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:01.860899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88ae0-8d70-435d-9fb2-d2dbdcb42c29 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:01.861331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88ae0-8d70-435d-9fb2-d2dbdcb42c29 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:01.861480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88ae0-8d70-435d-9fb2-d2dbdcb42c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:01.936691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-23d88ae0-8d70-435d-9fb2-d2dbdcb42c29 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:01.937945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-23d88ae0-8d70-435d-9fb2-d2dbdcb42c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:01.979553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:02.033978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:02.187041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:02.188574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:02.333009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333bcc-1744-4c8e-bd7e-22a5d01dd396 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:02.385725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:02.442640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:02.443282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:02.443569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:02.443844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:02.445510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:02.532507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:02.534128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fdb0f-961a-41df-8e5f-414cef5b134d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:02.552204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 726/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:03.438269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:03.438799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:03.438799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:03.440149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:03.445257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce101122-b6d6-4c54-82a2-1f22bdd77ab5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:03.445684 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce101122-b6d6-4c54-82a2-1f22bdd77ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:03.446327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce101122-b6d6-4c54-82a2-1f22bdd77ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:03.520279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ce101122-b6d6-4c54-82a2-1f22bdd77ab5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:03.521511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ce101122-b6d6-4c54-82a2-1f22bdd77ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:03.523506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:03.623700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:03.776152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:03.777714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:03.921399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c16e-7c66-4110-878f-838ca4883356 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:03.947567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:04.034708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:04.035276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:04.035556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:04.035828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:04.037500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:04.127979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:04.129635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb6898-f056-470b-bd5c-805fa38d88a2 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:04.147848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 713/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:42:06.081432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:06.082009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:06.082009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:06.083333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:06.088799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef4a3-6e66-4ec5-a338-f2baf5b02504 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:06.089238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef4a3-6e66-4ec5-a338-f2baf5b02504 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:06.089385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef4a3-6e66-4ec5-a338-f2baf5b02504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:06.128166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:06.154087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0b9ef4a3-6e66-4ec5-a338-f2baf5b02504 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:06.155274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0b9ef4a3-6e66-4ec5-a338-f2baf5b02504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:06.251293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:06.290580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:06.292156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:06.394810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c73beb-081a-413e-a1f2-f93ab21ea08f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:06.426775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:06.473396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:06.474032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:06.474310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:06.474581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:06.476195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:06.582564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:06.584004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79f131-8431-4acf-9d95-be385b9a02ef service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:06.603359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 729/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:08.788293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:08.788895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:08.788895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:08.790863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:08.796229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7389bb-c377-4f99-bc9d-91450b31cb43 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:08.796662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7389bb-c377-4f99-bc9d-91450b31cb43 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:08.796769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7389bb-c377-4f99-bc9d-91450b31cb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:08.860013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ee7389bb-c377-4f99-bc9d-91450b31cb43 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:08.861242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee7389bb-c377-4f99-bc9d-91450b31cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:08.874127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:08.956500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:08.995585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:08.997128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:09.097810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddfe3b-74fe-4d15-9e95-56d6a686e025 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:09.110751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:09.171116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:09.171761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:09.172040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:09.172318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:09.174130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:09.279802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:09.281246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8443e6-0297-461e-950e-82141badf2f7 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:09.299583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 716/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:11.718282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:11.718853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:11.718853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:11.720190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:11.725396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95cae-1573-442f-ba7d-9c78d8c9c7fd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:11.725828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95cae-1573-442f-ba7d-9c78d8c9c7fd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:11.725941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f95cae-1573-442f-ba7d-9c78d8c9c7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:11.773276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:11.791009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a8f95cae-1573-442f-ba7d-9c78d8c9c7fd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:11.792223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a8f95cae-1573-442f-ba7d-9c78d8c9c7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:11.889268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:11.928430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:11.931167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:12.034564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac593a5-5315-41a5-8d16-ba40746382eb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:12.074205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:12.108261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:12.108898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:12.109209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:12.109482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:12.111096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:12.217427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:12.218862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a55d71-fb32-48d5-87f4-c4332e42c9ae service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:12.237898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 732/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:14.375238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:14.375790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:14.375790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:14.377215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:14.382509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ea3f-1161-4a99-8051-339c9f9bea69 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:14.383077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ea3f-1161-4a99-8051-339c9f9bea69 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:14.383190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ea3f-1161-4a99-8051-339c9f9bea69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:14.446213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2940ea3f-1161-4a99-8051-339c9f9bea69 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:14.447335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2940ea3f-1161-4a99-8051-339c9f9bea69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:14.467776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:14.542727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:14.582003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:14.583538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:14.686012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e35d17-967f-4f0f-bfe3-5c883e8dabc7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:14.705886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:14.760878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:14.761555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:14.761832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:14.762105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:14.763723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:14.869526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:14.870947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297d8d2-9f21-4d54-9cf2-cc1b8c5728dd service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:14.888859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 719/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:42:16.981620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:16.982182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:16.982182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:16.983573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:16.989014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70369df0-198c-4bfb-b8cc-b1ed8423dd63 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:16.989462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70369df0-198c-4bfb-b8cc-b1ed8423dd63 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:16.989602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70369df0-198c-4bfb-b8cc-b1ed8423dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:17.050947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:17.053668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-70369df0-198c-4bfb-b8cc-b1ed8423dd63 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:17.056387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-70369df0-198c-4bfb-b8cc-b1ed8423dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:17.151540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:17.190207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:17.191780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:17.296989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1231a-5b17-4b2d-b2d5-97c24d3e9d3a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:17.346504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:17.372199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:17.372867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:17.373198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:17.378368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:17.378368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:17.482375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:17.483814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046de21e-4c54-4945-8a3c-94177bc045b8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:17.502782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 735/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:19.552576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:19.553184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:19.553184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:19.554734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:19.560081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63350886-1f22-44b4-a91c-0cdeac5c3c1c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:19.560555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63350886-1f22-44b4-a91c-0cdeac5c3c1c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:19.560652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63350886-1f22-44b4-a91c-0cdeac5c3c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:19.624934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-63350886-1f22-44b4-a91c-0cdeac5c3c1c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:19.626169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-63350886-1f22-44b4-a91c-0cdeac5c3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:19.655233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:19.723446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:19.761641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:19.763199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:19.870840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615303e-4766-45a5-8411-e03dd5224431 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:19.900683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:19.946292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:19.947121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:19.947654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:19.948302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:19.950274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:20.058513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:20.059856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb5c21-6eb2-4005-9111-70c96f12fed4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:20.078959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 722/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:22.148832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:22.149404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:22.149404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:22.150796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:22.156220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0f18-edd7-4fab-b964-62d1dc9d579c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:22.156672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0f18-edd7-4fab-b964-62d1dc9d579c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:22.156772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0f18-edd7-4fab-b964-62d1dc9d579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:22.222421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f64c0f18-edd7-4fab-b964-62d1dc9d579c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:22.223627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f64c0f18-edd7-4fab-b964-62d1dc9d579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:22.231920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:22.318650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:22.357170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:22.358695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:22.460077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec4973-d225-4000-9394-d84060794ff3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:22.515094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:22.532607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:22.533307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:22.533586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:22.533855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:22.535475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:22.760919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:22.762290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc986c6f-a830-44db-a4f2-73c58b05c715 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:22.779915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 738/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:24.817334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:24.817924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:24.817924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:24.819258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:24.825527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80858f2-57e7-4423-ab96-611c606b60fd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:24.825935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80858f2-57e7-4423-ab96-611c606b60fd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:24.826109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80858f2-57e7-4423-ab96-611c606b60fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:24.889070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f80858f2-57e7-4423-ab96-611c606b60fd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:24.890281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f80858f2-57e7-4423-ab96-611c606b60fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:24.927089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:24.983838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:42:25.021460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:25.023117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:25.127281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08cc45-278b-4840-a338-cfda83fd33cc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:25.165539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:25.201869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:25.202517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:25.202795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:25.203073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:25.204695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:25.312233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:25.313705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf638-3ba0-4a1b-a53c-70e2faaf634f service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:25.332501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 725/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:27.340099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:27.340625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:27.340625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:27.342105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:27.347467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3934-6b95-4f12-99ac-c38684b302aa None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:27.347891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3934-6b95-4f12-99ac-c38684b302aa None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:27.348057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3934-6b95-4f12-99ac-c38684b302aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:27.412439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c2da3934-6b95-4f12-99ac-c38684b302aa None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:27.413923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c2da3934-6b95-4f12-99ac-c38684b302aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:27.432373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:27.510387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:27.546856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:27.548361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:27.650405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619d2f0-7777-441c-8300-29bcec62c0f6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:27.715558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:27.721931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:27.728178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:27.728178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:27.728178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:27.728178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:27.833450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:27.834864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219030e0-b21b-431a-8df5-0b1a5f42ded7 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:27.853352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 741/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:29.874558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:29.875081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:29.875198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:29.876486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:42:29.881890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6a63-6607-4369-bd2b-d011219a719f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:29.882345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6a63-6607-4369-bd2b-d011219a719f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:29.882649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6a63-6607-4369-bd2b-d011219a719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:29.946830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba0f6a63-6607-4369-bd2b-d011219a719f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:29.948037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba0f6a63-6607-4369-bd2b-d011219a719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:29.995442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:30.043435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:42:30.080159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:30.081805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:30.188971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaa5a0-4546-4ea3-933e-b628b28823f8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:30.236938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:30.262492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:30.263147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:30.263428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:30.263704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:30.266066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:30.374066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:30.375482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b40457-c480-4164-8d11-8cb04d0e414d service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:30.393441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 728/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:40.041587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=22 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:40.391152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=66 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:49.920677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e945d1-335e-43f4-9aec-2c539b22d2ee None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:49.921275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e945d1-335e-43f4-9aec-2c539b22d2ee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:49.921275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e945d1-335e-43f4-9aec-2c539b22d2ee None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:49.922661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 743/1471] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:49.928089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a945-e4c3-4e75-b2ca-c33575f97ee6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:49.928545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a945-e4c3-4e75-b2ca-c33575f97ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:49.928662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a945-e4c3-4e75-b2ca-c33575f97ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:49.992336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7226a945-e4c3-4e75-b2ca-c33575f97ee6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:49.993587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7226a945-e4c3-4e75-b2ca-c33575f97ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:50.091690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 729/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:50.139873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:50.140163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:50.140456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:50.141829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 744/1473] 10.222.0.47 () {54 vars in 960 bytes} [Sat Jul 16 12:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:50.147272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06470ddc-37dc-4295-a4af-fd48d12fd4d2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:50.147565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06470ddc-37dc-4295-a4af-fd48d12fd4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:50.147845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06470ddc-37dc-4295-a4af-fd48d12fd4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:50.221712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-06470ddc-37dc-4295-a4af-fd48d12fd4d2 None None] MFA Rules not processed for user `eff6db7c70f64a649a64313389f7e9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:50.222942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-06470ddc-37dc-4295-a4af-fd48d12fd4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:50.244357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:50.299995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 730/1474] 10.222.0.47 () {58 vars in 1057 bytes} [Sat Jul 16 12:42:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:50.307633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:50.309426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:50.408776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb260e-8bc5-4e7d-aef2-8df0d2f77ec5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:50.479923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:50.480494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:50.480773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:50.481122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:50.482694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:50.533831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:50.589328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:50.590749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330018-6dbd-470f-a115-c103bc1d9cae service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:50.609293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 745/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:52.726250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:52.726819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:52.726819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:52.728188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 731/1476] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:52.733515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5e20-47f5-4245-adee-e23a5bc5d830 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:52.733949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5e20-47f5-4245-adee-e23a5bc5d830 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:52.734069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5e20-47f5-4245-adee-e23a5bc5d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:52.796536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-389c5e20-47f5-4245-adee-e23a5bc5d830 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:52.797763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-389c5e20-47f5-4245-adee-e23a5bc5d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:52.892072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 746/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:52.915865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:52.917454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:53.042686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d23b5a-5377-45f1-b9f2-740dd43ca279 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:53.091793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:53.136614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:53.137385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:53.137755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:53.137991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:53.138724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:53.140471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:42:53.140471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:42:53.140471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:42:53.141659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c4aa9-c781-47a7-8cb0-d27401726b83 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:53.160368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 732/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Sat Jul 16 12:42:52 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:42:55.187289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:55.187742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:55.187864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:55.189244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 747/1479] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:55.194709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af258f-e6bf-46e2-b82b-70b204531c76 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:55.195133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af258f-e6bf-46e2-b82b-70b204531c76 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:55.195261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af258f-e6bf-46e2-b82b-70b204531c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:55.259514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a9af258f-e6bf-46e2-b82b-70b204531c76 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:55.260750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9af258f-e6bf-46e2-b82b-70b204531c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:55.356430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 733/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:55.399774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:55.401383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:55.410874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:55.562673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e840d-7926-4e06-b25d-22cfce0d900a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:55.641078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:55.677066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:55.678299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:55.678583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:55.678854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:55.680491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:55.769073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:55.770658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46ad4-cc52-472d-ab5f-b043f6f85d52 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:55.788363 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 748/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:42:57.977242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:57.977809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:57.977809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:57.979188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 734/1482] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:42:57.984483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20728f-aab0-4431-84f3-13c4fce0a96d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:57.984899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20728f-aab0-4431-84f3-13c4fce0a96d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:57.985065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20728f-aab0-4431-84f3-13c4fce0a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:58.049871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1a20728f-aab0-4431-84f3-13c4fce0a96d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:42:58.063156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1a20728f-aab0-4431-84f3-13c4fce0a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:58.100101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:58.150276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 749/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:42:58.187373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:42:58.189026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:58.212755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:58.291385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157cc77-822a-4e51-bd73-c1d2ebae59f4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:42:58.366216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:42:58.366857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:42:58.367140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:42:58.367408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:42:58.369083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:42:58.446323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:42:58.480627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:42:58.482091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f0f8b-bc3f-4536-ada2-80ba88cf97ad service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:42:58.500164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 735/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:43:00.563677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:00.564266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:00.564266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:00.565690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 750/1485] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:43:00.571062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b305c-c106-42f4-b4a1-616dc1602de3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:00.571491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b305c-c106-42f4-b4a1-616dc1602de3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:00.571735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b305c-c106-42f4-b4a1-616dc1602de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:00.637844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ab4b305c-c106-42f4-b4a1-616dc1602de3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:43:00.639050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab4b305c-c106-42f4-b4a1-616dc1602de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:00.736128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 736/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:43:00.774743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:43:00.776271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:00.877018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b01e71-7039-40a6-bf60-29f382947c75 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:43:00.948232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:00.950000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:43:00.950644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:00.950929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:00.951205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:00.952851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:01.059193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:43:01.060620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26834b2-079a-4584-a06d-4d03b9261832 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:43:01.079752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 751/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:43:03.242158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:03.242705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:03.242705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:03.244112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 737/1488] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:43:03.249572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543dbee-6fd6-479c-ab59-9c92e4edd2e7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:03.249980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543dbee-6fd6-479c-ab59-9c92e4edd2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:03.250099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543dbee-6fd6-479c-ab59-9c92e4edd2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:03.314511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c543dbee-6fd6-479c-ab59-9c92e4edd2e7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:43:03.315695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c543dbee-6fd6-479c-ab59-9c92e4edd2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:03.368600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:03.412846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 752/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:43:03.453959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:43:03.455515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:03.491171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:03.562128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2841c-5982-4870-8879-a39e967ad3f4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:43:03.639302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:43:03.639958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:03.640242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:03.640523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:03.642337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:03.727580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:03.760061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:43:03.761529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb548cc-e9ea-4728-9d25-c8223758fd68 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:43:03.786376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 738/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:43:06.228710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:06.229258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:06.229357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:06.230653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 753/1491] 10.222.0.47 () {54 vars in 926 bytes} [Sat Jul 16 12:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:43:06.235817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ee20-f83c-476c-b767-9f033ff1c2a3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:06.236196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ee20-f83c-476c-b767-9f033ff1c2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:06.236378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ee20-f83c-476c-b767-9f033ff1c2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:06.299368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-24a1ee20-f83c-476c-b767-9f033ff1c2a3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:43:06.300596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-24a1ee20-f83c-476c-b767-9f033ff1c2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:06.396991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 739/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Sat Jul 16 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:43:06.438054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:43:06.439605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:06.540512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560b843-8dbe-4f50-bc18-49e9db0591ab None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:43:06.612191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:43:06.612839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:43:06.613154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:43:06.613436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:43:06.615735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:43:06.624625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:06.723528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:43:06.725603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085309b-7052-4dfb-8a58-18cf77e1654a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:43:06.743337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 754/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:43:16.395113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=82 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:43:16.740925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=50 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:45:03.188403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.188938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.189063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.267487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:45:03.268697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:03.370933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 740/1494] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:45:03.378927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.379428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.379533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.398058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:45:03.399853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:03.409153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 755/1495] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:45:03.413643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.423233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf047f4-05a8-466e-a764-78b113fda51c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.428682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.429367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.429647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.429921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.430692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:45:03.430948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.431939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.431939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:45:03.431939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:45:03.432831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.444823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 741/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:45:03.452992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.461977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324e7db-579a-4789-861a-42204d11fba4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.466487 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.467117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.467395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.467668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.468365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.470106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.470106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:45:03.470106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:45:03.471927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.591405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 756/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:45:03 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:45:03.596311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.606375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a7b9-6db0-44c2-b960-1eb0dd9bb392 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.657813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:45:03.665113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.665746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.666040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.666310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.667017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.668029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.668029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:45:03.668029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:45:03.669255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.786688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 742/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:45:03 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:45:03.790707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.800838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46557a25-d3a8-4566-bf82-09bedcc505fe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.805465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.806116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.806404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.806676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.807491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.808501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.808501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:45:03.808501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:45:03.809413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.823673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 757/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:45:03.827800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.838952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827eeab-2ae3-4c2a-b6e9-b4bc5ca5e79f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.843747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.844412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.844699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.844976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.846485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.847497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.847497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:45:03.847497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:45:03.848399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ace00c-c34f-43b7-a629-98f87bf6ae89 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.861903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 743/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:45:03.866060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:03.875877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0438b-23a7-4ee8-9f27-dd3b5a4a5433 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:03.881095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:03.884565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.884565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.884565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] PATH_INFO: `/v3/projects/bcdb2a6fe1fc4517ad6db7153066bf7b/users/5bcac024ab3144e493d68fb12cbe995f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:03.897477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdb2a6fe1fc4517ad6db7153066bf7b, user_id=5bcac024ab3144e493d68fb12cbe995f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:03.898499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:03.898499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:45:03.898499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:45:03.900401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc100a-3eb3-4220-b502-57db941b7ae7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:03.974684 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 758/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:45:03 2022] PUT /identity/v3/projects/bcdb2a6fe1fc4517ad6db7153066bf7b/users/5bcac024ab3144e493d68fb12cbe995f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:45:03.986591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed899eb-e693-4d09-9841-f36003946f07 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:03.986874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed899eb-e693-4d09-9841-f36003946f07 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:03.987154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed899eb-e693-4d09-9841-f36003946f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.014248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0ed899eb-e693-4d09-9841-f36003946f07 None None] MFA Rules not processed for user `5bcac024ab3144e493d68fb12cbe995f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:45:04.015450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ed899eb-e693-4d09-9841-f36003946f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:04.080544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 744/1502] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:45:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:45:04.094046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff81108-d1c9-42cd-9b03-fbf52e8e8993 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.094338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff81108-d1c9-42cd-9b03-fbf52e8e8993 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.094614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff81108-d1c9-42cd-9b03-fbf52e8e8993 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.096111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 759/1503] 10.222.0.47 () {54 vars in 963 bytes} [Sat Jul 16 12:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:45:04.101129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca8b7-b1d3-4b6b-874e-962b2722ebf3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.101426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca8b7-b1d3-4b6b-874e-962b2722ebf3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.101728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca8b7-b1d3-4b6b-874e-962b2722ebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.159010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-44aca8b7-b1d3-4b6b-874e-962b2722ebf3 None None] MFA Rules not processed for user `d10255724102414ca184fac122fd1e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:45:04.160329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-44aca8b7-b1d3-4b6b-874e-962b2722ebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:04.237557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 745/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Sat Jul 16 12:45:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:45:04.245310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff81108-d1c9-42cd-9b03-fbf52e8e8993 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:04.256560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff81108-d1c9-42cd-9b03-fbf52e8e8993 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:04.260206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:04.260833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.261140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.261415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.270396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:04.271793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:04.280511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 760/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:45:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:45:04.332048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b757f3-b287-426f-bbee-420e0eb9c2d2 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.332340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b757f3-b287-426f-bbee-420e0eb9c2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.332622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b757f3-b287-426f-bbee-420e0eb9c2d2 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.334127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 746/1506] 10.222.0.47 () {52 vars in 829 bytes} [Sat Jul 16 12:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:45:04.337620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:04.398330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff8b6-b475-4adf-bdcd-bb530dabf608 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:04.402973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:04.403826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.404104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.404376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] PATH_INFO: `/v3/users/5bcac024ab3144e493d68fb12cbe995f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.405587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcac024ab3144e493d68fb12cbe995f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:04.406597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:04.406597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:45:04.406597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:45:04.407889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:04.781329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcac024ab3144e493d68fb12cbe995f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:04.794244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcac024ab3144e493d68fb12cbe995f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:04.813475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcac024ab3144e493d68fb12cbe995f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:04.814338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcac024ab3144e493d68fb12cbe995f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:04.818753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:45:04.894685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcac024ab3144e493d68fb12cbe995f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:04.901353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 761/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:45:04 2022] DELETE /identity/v3/users/5bcac024ab3144e493d68fb12cbe995f => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:45:04.905862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:04.906145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:04.906445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:04.926377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:45:04.927646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:04.937281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:45:04.988878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 747/1508] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:45:04.998558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:04.999981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:05.080513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936996b-85db-4e1e-b3d1-f07fd6bf3457 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:05.084287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:05.084941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:05.085249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:05.085519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:05.094911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:05.097107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc0ff0-c208-4174-aed6-07ccc6666062 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:05.106936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 762/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:45:05.593098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:45:05.596027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:45:05.606777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe6fba-f0fe-4cf9-9dbd-cc84f238d0f2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:45:05.613590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:45:05.614808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:45:05.615097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:45:05.615482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] PATH_INFO: `/v3/projects/bcdb2a6fe1fc4517ad6db7153066bf7b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:45:05.616578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdb2a6fe1fc4517ad6db7153066bf7b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:45:05.617904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:45:05.617904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:45:05.617904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:45:05.619316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:45:05.758031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdb2a6fe1fc4517ad6db7153066bf7b', 'request_id': 'req-5145196d-99f2-4fe5-b8d2-d979e0183638'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:05.771300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdb2a6fe1fc4517ad6db7153066bf7b', 'request_id': 'req-5145196d-99f2-4fe5-b8d2-d979e0183638'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:05.853883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdb2a6fe1fc4517ad6db7153066bf7b', 'request_id': 'req-5145196d-99f2-4fe5-b8d2-d979e0183638'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:05.866041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] The token cache is being invalidate because project bcdb2a6fe1fc4517ad6db7153066bf7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:45:05.866325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:05.867051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5145196d-99f2-4fe5-b8d2-d979e0183638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:45:05.868592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 748/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:45:05 2022] DELETE /identity/v3/projects/bcdb2a6fe1fc4517ad6db7153066bf7b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:45:15.102558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:45:15.862925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:46:46.189514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:46.190004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:46.190165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:46.191488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:46:46.196571 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c05f-07c8-485c-aa4f-cef3f59c89c3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:46.197005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c05f-07c8-485c-aa4f-cef3f59c89c3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:46.197139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c05f-07c8-485c-aa4f-cef3f59c89c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:46.215242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2903c05f-07c8-485c-aa4f-cef3f59c89c3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:46.216482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2903c05f-07c8-485c-aa4f-cef3f59c89c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:46.275875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:46:46.437127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:46:46.439151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:46.539649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176244a-c5c5-48e0-8f6b-dfc9c7794ada None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:46:46.545099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:46:46.545738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:46.546020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:46.546294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:46.554315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:46:46.555880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24449886-dc7a-466c-b5df-abde009d17db service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:46:46.565470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 764/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:46:47.397907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1eec9-a2e5-40b7-8b0e-5791e8ac9f28 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:47.398434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1eec9-a2e5-40b7-8b0e-5791e8ac9f28 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:47.398434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1eec9-a2e5-40b7-8b0e-5791e8ac9f28 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:47.399793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:46:47.404761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a86d-1a00-4387-953e-15117cfb2ca3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:47.405225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a86d-1a00-4387-953e-15117cfb2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:47.405329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a86d-1a00-4387-953e-15117cfb2ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:47.427350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-25d0a86d-1a00-4387-953e-15117cfb2ca3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:47.428543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-25d0a86d-1a00-4387-953e-15117cfb2ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:47.436358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:46:47.588161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1eec9-a2e5-40b7-8b0e-5791e8ac9f28 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:46:47.603188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1eec9-a2e5-40b7-8b0e-5791e8ac9f28 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:46:47.608607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:46:47.609265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:47.609548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:47.609818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:47.618759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:46:47.620339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee685810-16f2-40ca-9244-82fafd1a2a83 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:46:47.629818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 751/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:46:48.665949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213bdd3-2fc4-4a28-905a-02fdaeaecc85 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:48.666534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213bdd3-2fc4-4a28-905a-02fdaeaecc85 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:48.666534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213bdd3-2fc4-4a28-905a-02fdaeaecc85 None None] PATH_INFO: `/v3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:48.667812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 766/1517] 10.222.0.26 () {54 vars in 925 bytes} [Sat Jul 16 12:46:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:46:48.673943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809a1db-43a9-49b0-a33b-bc52126e2c57 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:48.674911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809a1db-43a9-49b0-a33b-bc52126e2c57 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:48.675123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809a1db-43a9-49b0-a33b-bc52126e2c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:48.845776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8809a1db-43a9-49b0-a33b-bc52126e2c57 None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:48.847515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8809a1db-43a9-49b0-a33b-bc52126e2c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:48.918735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 752/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jul 16 12:46:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:46:52.947765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e0186-95a4-4515-9cb3-6ba3e625d8b7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:52.948286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e0186-95a4-4515-9cb3-6ba3e625d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:52.948286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e0186-95a4-4515-9cb3-6ba3e625d8b7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:52.949616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 767/1519] 10.222.0.47 () {54 vars in 960 bytes} [Sat Jul 16 12:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:46:52.954604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d424f-2540-45e9-a975-9d1e16a1b5dc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:52.955068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d424f-2540-45e9-a975-9d1e16a1b5dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:52.955192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d424f-2540-45e9-a975-9d1e16a1b5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:52.973560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0d0d424f-2540-45e9-a975-9d1e16a1b5dc None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:52.974741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0d0d424f-2540-45e9-a975-9d1e16a1b5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:52.983088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 753/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Sat Jul 16 12:46:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:46:52.990602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e0186-95a4-4515-9cb3-6ba3e625d8b7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:46:53.003915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e0186-95a4-4515-9cb3-6ba3e625d8b7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:46:53.008662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:46:53.009303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:53.009586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:53.009852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:53.017866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:46:53.020004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcab8ed-9bf3-4d02-b6fe-50f4e4ec3d6d service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:46:53.029273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 768/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:46:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:46:58.855458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684df0-a223-4c9a-9e1d-493dfa041100 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:58.856029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684df0-a223-4c9a-9e1d-493dfa041100 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:58.856029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684df0-a223-4c9a-9e1d-493dfa041100 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:58.857323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:46:58.862014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0350-e4b5-40de-86b3-d36a0bcdecfc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:58.862467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0350-e4b5-40de-86b3-d36a0bcdecfc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:58.862573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0350-e4b5-40de-86b3-d36a0bcdecfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:58.880728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-726f0350-e4b5-40de-86b3-d36a0bcdecfc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:58.881950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-726f0350-e4b5-40de-86b3-d36a0bcdecfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:58.889694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:46:59.045281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684df0-a223-4c9a-9e1d-493dfa041100 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:46:59.059807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684df0-a223-4c9a-9e1d-493dfa041100 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:46:59.065384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:46:59.066768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:59.067044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:59.067321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:59.075418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:46:59.077022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba9d4-2a00-40ce-a6e1-34c372c3380a service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:46:59.086377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 755/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:46:59.905763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aec4-209c-4316-a145-5f7e1cf90d0e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:59.906330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aec4-209c-4316-a145-5f7e1cf90d0e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:59.906330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aec4-209c-4316-a145-5f7e1cf90d0e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:59.907671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 12:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:46:59.913216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64329-3721-43de-a420-08767ac22cc2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:46:59.913641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64329-3721-43de-a420-08767ac22cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:46:59.913752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64329-3721-43de-a420-08767ac22cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:46:59.931438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2bd64329-3721-43de-a420-08767ac22cc2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:46:59.932622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2bd64329-3721-43de-a420-08767ac22cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:46:59.940597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 12:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:47:00.094501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456aec4-209c-4316-a145-5f7e1cf90d0e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:47:00.109107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456aec4-209c-4316-a145-5f7e1cf90d0e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:47:00.114396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:47:00.115016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:00.115293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:00.115558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:00.123430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:47:00.124970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1d128-ea9e-4b46-9ae4-5c7b53e6d94e service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:47:00.135144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 771/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:47:03.921875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:03.922435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:03.922435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:03.923879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:47:03.929688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d279a-ed99-4663-bc8e-34e3f2fbc6d2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:03.930141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d279a-ed99-4663-bc8e-34e3f2fbc6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:03.930252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d279a-ed99-4663-bc8e-34e3f2fbc6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:03.948427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-505d279a-ed99-4663-bc8e-34e3f2fbc6d2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:47:03.949672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-505d279a-ed99-4663-bc8e-34e3f2fbc6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:03.957600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:47:03.996268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:47:03.997988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:04.026911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:47:04.056307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe6988-2b03-4dd4-af1d-a11adbe73b64 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:47:04.059968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:47:04.060813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:04.061393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:04.061671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:04.070746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:47:04.072157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dba9b-f18b-4280-9cb1-20c20fc66fdb service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:47:04.081480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 758/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:47:06.101265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045c49-26f8-48af-89f8-348629d9967d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:06.101799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045c49-26f8-48af-89f8-348629d9967d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:06.101799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045c49-26f8-48af-89f8-348629d9967d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:06.103103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:47:06.108203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7ec6-4d4d-4356-8917-e1328afc2b2f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:06.108634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7ec6-4d4d-4356-8917-e1328afc2b2f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:06.108815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7ec6-4d4d-4356-8917-e1328afc2b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:06.127030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-61cc7ec6-4d4d-4356-8917-e1328afc2b2f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:47:06.128857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-61cc7ec6-4d4d-4356-8917-e1328afc2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:06.136679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:47:06.172415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78045c49-26f8-48af-89f8-348629d9967d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:47:06.183944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78045c49-26f8-48af-89f8-348629d9967d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:47:06.187439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:47:06.188326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:06.188605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:06.188876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:06.198485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:47:06.199899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199912d3-bcff-4b9a-bc1c-7b2cd71678fc service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:47:06.209492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 774/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:47:09.764591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:09.765157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:09.765157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:09.766452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:47:09.771634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f988-54c2-4f30-a45d-cd58947d9a4b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:09.772072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f988-54c2-4f30-a45d-cd58947d9a4b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:09.772190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f988-54c2-4f30-a45d-cd58947d9a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:09.789861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e530f988-54c2-4f30-a45d-cd58947d9a4b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:47:09.791037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e530f988-54c2-4f30-a45d-cd58947d9a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:09.799102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:47:10.101149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:47:10.102826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:10.158816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399898-f70d-4974-9112-c88f69d2c10f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:47:10.162770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:47:10.163406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:10.163689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:10.163963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:10.172867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:47:10.174323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ab198-65bc-437e-bdaa-7bf9e58c0c5d service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:47:10.184259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 761/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:47:12.194305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:12.194748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:12.194862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:12.196218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 12:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:47:12.201298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf128e-c26c-4d70-b35a-b1c3227a60d1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:12.201699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf128e-c26c-4d70-b35a-b1c3227a60d1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:12.201855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf128e-c26c-4d70-b35a-b1c3227a60d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:12.220560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-55cf128e-c26c-4d70-b35a-b1c3227a60d1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:47:12.222119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-55cf128e-c26c-4d70-b35a-b1c3227a60d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:12.229814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 12:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:47:12.528398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:47:12.530121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:47:12.543063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdcf842-da1e-416e-9319-a8897946f7bb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:47:12.546606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:47:12.547233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:47:12.547513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:47:12.547788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:47:12.557868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:47:12.559288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f27a1-6390-4065-9346-a9037af0eae6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:47:12.569765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 777/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:47:22.215427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=47 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:47:22.565467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=82 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:41.156057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.156775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.156775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.179512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.180752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.195553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 763/1540] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.212295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471f5a-a6a2-4ae2-917b-3580d00d3f73 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.213319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471f5a-a6a2-4ae2-917b-3580d00d3f73 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.213813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471f5a-a6a2-4ae2-917b-3580d00d3f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.240039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-75471f5a-a6a2-4ae2-917b-3580d00d3f73 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.241614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-75471f5a-a6a2-4ae2-917b-3580d00d3f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.256811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 778/1541] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.261300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.274015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b9035-000c-4209-a2bd-950f18036fa9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.282674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.284334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.284826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.285332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.286278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:41.286780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.288704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d3d0e-6608-44bd-9afb-4a04cc17b998 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.303811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.304439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.304924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.309062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 764/1542] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:41.328997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d170-4177-4f25-8522-b171486576c7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.329311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d170-4177-4f25-8522-b171486576c7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.329812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d170-4177-4f25-8522-b171486576c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.333729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.335048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.352366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 779/1543] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:48:41.355491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.370095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c9d9d170-4177-4f25-8522-b171486576c7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.370707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3cc47-0bac-4a9b-9466-eaaee2b60821 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.371322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c9d9d170-4177-4f25-8522-b171486576c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.382702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.383698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.384146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.384580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.385452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.387525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42212d1c-dbf5-499b-abf3-f4b4aa4a46db admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.394821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 765/1544] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:48:41.396950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9710-ee9f-476f-946f-9fd98e58776f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.397428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9710-ee9f-476f-946f-9fd98e58776f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.397862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9710-ee9f-476f-946f-9fd98e58776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.431597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b1fd9710-ee9f-476f-946f-9fd98e58776f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.433105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b1fd9710-ee9f-476f-946f-9fd98e58776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.442486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 766/1545] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.444100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba159db-2075-4a35-bf98-42cc05f274cb None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.444562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba159db-2075-4a35-bf98-42cc05f274cb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.445009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba159db-2075-4a35-bf98-42cc05f274cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.465368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0ba159db-2075-4a35-bf98-42cc05f274cb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.466609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ba159db-2075-4a35-bf98-42cc05f274cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.475365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 767/1546] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.479030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.479030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.479030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.496277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.497985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.505965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 780/1547] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/projects => generated 405 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:41.506532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 768/1548] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.507570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.508392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd06-d1f2-439f-a18a-21ff3b373681 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.509016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd06-d1f2-439f-a18a-21ff3b373681 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.518961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd06-d1f2-439f-a18a-21ff3b373681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.519112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d51691-c366-43bb-93f0-4ae82e49cc5c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.529984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f675fd06-d1f2-439f-a18a-21ff3b373681 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.531609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f675fd06-d1f2-439f-a18a-21ff3b373681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.583276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.583940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.584223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.584498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.585254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:41.585562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.587174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fd3c7-5a15-4261-b4b6-eda57584637b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.594890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 781/1549] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.597117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948a44-d0ad-40b7-bd8d-a43e2585a7fb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.597412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948a44-d0ad-40b7-bd8d-a43e2585a7fb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.597687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948a44-d0ad-40b7-bd8d-a43e2585a7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.608793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 769/1550] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:41.608793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.608793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.608793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.616393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-50948a44-d0ad-40b7-bd8d-a43e2585a7fb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.617612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-50948a44-d0ad-40b7-bd8d-a43e2585a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.625612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 782/1551] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.627196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.627490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.627662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.627862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.628808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.639038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 770/1552] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.639809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.649797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1b105-0e44-4fff-8b36-0d97d72164e3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.653166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.654405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.654560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.655255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.655532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.655801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.656481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.659099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.663367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 783/1553] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.667140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.675569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb49de-6839-4f99-9aed-8d0039c5ac52 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.680233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.680873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.681218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.681492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.682257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:41.682512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.683796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:48:41.683796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:48:41.683796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:48:41.684669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200b543-52e3-454a-8b0f-4b2322a71c21 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.696822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 784/1554] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:41.698078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e765e-0e1a-4598-ba9d-dcea5eff22ff None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.698377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e765e-0e1a-4598-ba9d-dcea5eff22ff None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.698650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e765e-0e1a-4598-ba9d-dcea5eff22ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.715835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-877e765e-0e1a-4598-ba9d-dcea5eff22ff None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.718866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-877e765e-0e1a-4598-ba9d-dcea5eff22ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.724983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 785/1555] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.726903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbd7d2-97f6-4178-8dde-89648e6cbdf1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.727190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbd7d2-97f6-4178-8dde-89648e6cbdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.727464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbd7d2-97f6-4178-8dde-89648e6cbdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.742667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 771/1556] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:41.743881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.745694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-73cbd7d2-97f6-4178-8dde-89648e6cbdf1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.746900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73cbd7d2-97f6-4178-8dde-89648e6cbdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.754934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f8900-a6dc-4c67-9e60-aabe15ce0bbe admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.756852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 786/1557] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.758193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226221f-253b-4362-9154-694e35295ebc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.763091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226221f-253b-4362-9154-694e35295ebc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.763091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226221f-253b-4362-9154-694e35295ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.763309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.763309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.763309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.763309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.763651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.763651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:48:41.763651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:48:41.763651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:48:41.763931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59e551-5945-4c78-97da-b3b4add5c120 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.779971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0226221f-253b-4362-9154-694e35295ebc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.781296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0226221f-253b-4362-9154-694e35295ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.789299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 787/1558] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.798026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.798026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.798026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.809449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.810572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.819080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 788/1559] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.819958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.829108 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280149e9-4704-40ca-ad3c-16ec7d9101a6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.833607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.834221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.834505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.834782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.835491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:41.835796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.837411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.848921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 789/1560] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:41.849785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.853584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 772/1561] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:41.853584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.862166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.862166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.862386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194870b-bdd5-486e-8a9b-dc024cf01183 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.873997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:41.875231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:41.918224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.918919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.919203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.919476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.920161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.922088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.925843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:41.941370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 773/1562] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:41.941370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.942708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:41.947784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409460ee-3e8c-4e67-bd88-593ddce0dc5c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.952477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.959158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.959158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.959158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.959158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.959158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.966308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 774/1563] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:41.967078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:41.976594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f094-02b0-4062-915e-5c41fc914aa4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:41.981237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:41.984907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:41.984907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:41.984907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:41.984907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:41.984907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:41.985363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:41.998102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 775/1564] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:41.998885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.008803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967822b-fb66-4be5-bee0-98ebe550d5de admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.013451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.016557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.016557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.016557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.016557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:42.016557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.017309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.028485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 776/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:42.029281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.040416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac95ee-aac5-43b6-997b-a3b3f0ffbb93 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.041358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 790/1566] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:41 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.042082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.100144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.100814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.101128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.105440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.105440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:42.105440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.105739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29137-87be-4379-a752-fd7bd9fcce7a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.105842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.107881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.108534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.108817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.109125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.109817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.112479 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.116982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 777/1567] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:42.117872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.128589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d641-feab-479a-a566-f495ae1ce42f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.133204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.133847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.134125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.134395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.135072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.136941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.189745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 791/1568] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.191599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.216731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 778/1569] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.217896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.228279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9970-0fbe-4732-ad6e-dec455f041b9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.254199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e799c4-239f-444d-b199-6ee249ed2881 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.259549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.260353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.260634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.260905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.261680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:42.262107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.263698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.275892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 792/1570] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:42.283873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.286652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10158aa-1b05-4185-a668-91eae17b7720 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.291254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.291879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.292154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.292426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.292544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.293136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.293377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.293732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.294079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.294884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.295003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.296588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.307732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 779/1571] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:42.308333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.318701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbaaf2-ba43-468d-a5b4-bf4e502d0032 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.323264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.323884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.325371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.325371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.325371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.327111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.366367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 793/1572] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/projects => generated 403 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.367206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.411807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:42.423348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 780/1573] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.428714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.429097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b4b3-72a6-4bbf-9e10-b4da079fb176 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.450296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:42.484751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.485466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.485759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.486581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.487271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.488299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:48:42.488299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:48:42.488299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:48:42.489539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.492098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a882d-18df-4019-bbed-b03f554c8ddd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.496784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.502312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.502312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.502312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.502312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.502312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.587471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 794/1574] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.588453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.598152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd6c-8579-46b2-8e3b-d4ed1310837f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.602795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.603434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.603721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.603998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.604677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.606107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 781/1575] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.606776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.606899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.662215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 795/1576] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 289 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.663091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.667773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6fadf-0ef5-4d54-802c-9067e5d468db admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.674318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.675068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7920f-9221-4416-8700-efed8169be6e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.675247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.675591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.675959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.676764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.678470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.679843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.680802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.680802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.681140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.681831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.683980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.692785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 782/1577] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:42.693669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.703734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17988e-3eae-41bd-9720-cdde4ed698a0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.708105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.708966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.709271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.709681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] PATH_INFO: `/v3/projects/605135d537dc488db139784d0ade2446/users/8af59cb05c684c2d9b2df315e04ef2d0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.724362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605135d537dc488db139784d0ade2446, user_id=8af59cb05c684c2d9b2df315e04ef2d0, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.725415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:48:42.725415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:48:42.725415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:48:42.727340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.738523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 796/1578] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/projects => generated 391 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.739326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.776348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 783/1579] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:42 2022] PUT /identity/v3/projects/605135d537dc488db139784d0ade2446/users/8af59cb05c684c2d9b2df315e04ef2d0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:42.777987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.801029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d4ca-a470-4ef2-8a09-f4f17fe162c2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.842883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b45554-3b84-47f8-abb3-56afcf7820a1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.843352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:42.848261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.848933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.849240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.849515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.850208 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.852372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.856395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.857087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.857510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.857792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.858478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.860456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.914019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 784/1580] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.914844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.919648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:42.925305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d88c-8b98-411b-9f68-8eb7d65038b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.930168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.930801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.931106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.931379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.932087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.933945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.945388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 797/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:42.945388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.945661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 785/1582] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:42.945661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.952953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852841-d41c-4daa-8697-797d14071339 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.957620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e097-24a8-485f-a985-b48a6003e3d3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.957767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.958237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.958519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.958792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.959477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.961490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249506e-698e-4d46-af68-c7024f6c6738 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.962517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.963281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.963688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.964055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.964867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.966610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:42.979843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 786/1583] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:42.980644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:42.990811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623edbfe-5fd9-406a-957c-187ee1c186be admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:42.995296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:42.995969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:42.996248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:42.996520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:42.997294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:42.998841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.008265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 787/1584] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.009156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.018565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c9409-9b85-4018-98e3-39abc6dc2fdf admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.023029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.023623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.023902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.024172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.024821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.026684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.037567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 798/1585] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:42 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:43.037567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.037567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.037567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.064691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] MFA Rules not processed for user `8af59cb05c684c2d9b2df315e04ef2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:43.065932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:43.090617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 788/1586] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/users => generated 296 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:43.094600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.104825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c4c26-d215-4895-8fe2-113a872121b4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.109558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.110211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.110496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.110773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.112397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.114046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.124654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 789/1587] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.125664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.133630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 799/1588] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:43.134704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.137770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dbd6a-e681-4b69-8fda-9064b76bac16 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.143043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.143677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.143992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.144274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.145167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.145354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e37f-7d48-4392-9edb-14fc01378efe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.146802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.150005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.150645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.150923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.151194 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.151917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.153515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.160461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 790/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.160461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.165395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 800/1590] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.165395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.172025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317138c-7f77-4451-8c47-b1b714480cf5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.175967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fc8b3-49a1-471e-8732-ad5a1c4beef1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.181438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.182083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.182385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.182692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.183616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.185334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.185684 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.186712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.187058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.187399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] PATH_INFO: `/v3/projects/f762c3c164904a3bb6e64985744c1765/users/3e12d902f94b40cea088dc6f24dcb444/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.196568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 791/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.197412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.206088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f762c3c164904a3bb6e64985744c1765, user_id=3e12d902f94b40cea088dc6f24dcb444, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.207170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcab311-2d73-4630-8c13-1780e334cfa9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.209063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.211865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.212515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.212874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.213250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.213971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.215528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.225860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 792/1592] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.226695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.236600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc216-8d05-4d2e-9c02-611fe78d8ad7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.241851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.246404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.246404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.246404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.246404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.246404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.256488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 793/1593] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.257150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.262683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 801/1594] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/f762c3c164904a3bb6e64985744c1765/users/3e12d902f94b40cea088dc6f24dcb444/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:43.269410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.269533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe9b9-cd3d-4587-9545-cfa114f422b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.326385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45a55-98e4-4971-9be1-06dd0b046bc2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.326768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.327439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.327759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.327992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.328713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.330505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.331009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.331661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.331938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.332214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.332900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.334818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.341237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 794/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.342019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.353471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e304ae5-c9c0-44f8-a4d4-4f7acca60f37 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.361942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.361942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.361942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.361942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] PATH_INFO: `/v3/projects/5a6b49c60dc845c5a74f3ab5d132740d/users/f1c789055f344bb59a86b6d4fe5cac91/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.374232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6b49c60dc845c5a74f3ab5d132740d, user_id=f1c789055f344bb59a86b6d4fe5cac91, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.378178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.428006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:43.435224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 802/1596] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/projects => generated 401 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:43.440211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.457415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 795/1597] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/5a6b49c60dc845c5a74f3ab5d132740d/users/f1c789055f344bb59a86b6d4fe5cac91/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:43.466080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.490092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:43.501300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060404e-c934-45d6-aaec-303496c62e60 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.521093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c9852-eb7f-4801-8868-bed7d002f95a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.526102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.527381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.527663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.527946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] PATH_INFO: `/v3/projects/858d6855277f465497c27313765e5d1a/users/46b5c80d8bf34965a30b9b67e9117996/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.569863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=858d6855277f465497c27313765e5d1a, user_id=46b5c80d8bf34965a30b9b67e9117996, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.572519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.581285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.581959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.582265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.582541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.583263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.584879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.594705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 803/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.596183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.605563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73956b-0d4f-49bf-9eb3-df0a88f726c2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.610208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.610840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.611643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.611917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.612615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.614207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec8ea1-f4c2-4a90-904b-eacd4c697d63 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.622188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 796/1599] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/858d6855277f465497c27313765e5d1a/users/46b5c80d8bf34965a30b9b67e9117996/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:43.623001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.627952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 804/1600] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.627952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.627952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.627952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.645554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:43.646752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:43.685143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266c87e-0dd2-487d-8c4e-a33665482a4b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.689837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.690492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.690772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.691042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.691789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.693402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.703245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 797/1601] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:43.713784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.714445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 805/1602] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:43.714445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.719046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04add1e-6840-4b11-8530-aff06a85e6c4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.720033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352ef2b-496a-4b16-9952-421fad9a6637 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.724609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.725538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.725823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.726078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.726249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] PATH_INFO: `/v3/projects/aa39d77033fe427296b7e5fa6a49f3d3/users/09eca28965ab4ddfbe614704e8e884d6/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.726731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.727011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.727285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.727970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.729964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9db65-f4a9-499e-9020-987d4b9607ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.743347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa39d77033fe427296b7e5fa6a49f3d3, user_id=09eca28965ab4ddfbe614704e8e884d6, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.746039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.822933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 806/1603] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/users => generated 307 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:43.833306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 798/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/aa39d77033fe427296b7e5fa6a49f3d3/users/09eca28965ab4ddfbe614704e8e884d6/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:43.833306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.833490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe04c-c008-43e7-a819-453a557de386 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.833490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe04c-c008-43e7-a819-453a557de386 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.833490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe04c-c008-43e7-a819-453a557de386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.843575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-849fe04c-c008-43e7-a819-453a557de386 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:43.846711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-849fe04c-c008-43e7-a819-453a557de386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:43.854462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:43.891825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b121-0ff2-4920-9b8b-3e4f135b8a3c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.897304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.898194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.898468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.898735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] PATH_INFO: `/v3/projects/3b76170e55944f5786c114512c695204/users/64a52dc9fb4a4290812e77c0c5bf8c44/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.908766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 807/1605] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:43.910355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.910710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.911163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.916648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b76170e55944f5786c114512c695204, user_id=64a52dc9fb4a4290812e77c0c5bf8c44, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.919555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.928609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:43.929845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:43.937867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 808/1606] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:43.938669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.947657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768073c-53e2-47a5-ab05-6a44f05ca312 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:43.952153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:43.953005 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:43.953322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:43.953594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] PATH_INFO: `/v3/projects/a3bb5db6207441a1b5030af42f291666/users/628e662a3b49438f9930a2d160658b7c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:43.965384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 799/1607] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/3b76170e55944f5786c114512c695204/users/64a52dc9fb4a4290812e77c0c5bf8c44/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:43.966256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:43.973063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bb5db6207441a1b5030af42f291666, user_id=628e662a3b49438f9930a2d160658b7c, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:43.975501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:43.976741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:44.028358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0e596-ca2d-4975-a86c-5cdeca37dc23 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.033804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.034778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.035064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.035339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] PATH_INFO: `/v3/projects/6e8a673b65b64fbc916da4ce4cdc9f81/users/f1c9a7f942c541a287da38a894f1d1f9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.040369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 809/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/a3bb5db6207441a1b5030af42f291666/users/628e662a3b49438f9930a2d160658b7c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:44.041285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.054015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8a673b65b64fbc916da4ce4cdc9f81, user_id=f1c9a7f942c541a287da38a894f1d1f9, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.056718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.096734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c1a9-dcb9-42f7-95fc-3be85e3fc560 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.100355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.100992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.101298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.101567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.124895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 800/1609] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:43 2022] PUT /identity/v3/projects/6e8a673b65b64fbc916da4ce4cdc9f81/users/f1c9a7f942c541a287da38a894f1d1f9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:44.129627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.165822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.167265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2b674-8f4a-415e-bf84-664d41e0503a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.177652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 810/1610] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.178942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.179224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.179495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.196530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea08617-37cc-4454-8670-e9a1c285a043 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.203025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.203544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.203966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.204102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.205383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.207422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.209181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] MFA Rules not processed for user `09eca28965ab4ddfbe614704e8e884d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:44.210395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.217979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 801/1611] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:44.218775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.232440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:44.275331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea0553-5a36-4cbf-8d1f-015013a18fdb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.282924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 811/1612] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:44.283697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.284068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.284688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.284688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.285070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.295285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.295677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55586-be48-434b-8350-aa2de6325f36 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.297110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec6bcc-4df9-4d15-8076-02ebb78a157a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.299801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.300617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.300944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.301318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.307474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 802/1613] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.308817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ce2cb-6fbc-458d-83af-e098a05c0ba3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.309131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ce2cb-6fbc-458d-83af-e098a05c0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.309409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ce2cb-6fbc-458d-83af-e098a05c0ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.312208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.315066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a9aa9-ae5f-49ac-ac36-cdcfbbc90742 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.326450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 812/1614] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.328106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.328392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.328694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.328984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2f4ce2cb-6fbc-458d-83af-e098a05c0ba3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:44.331088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2f4ce2cb-6fbc-458d-83af-e098a05c0ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.339492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 803/1615] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:44.343820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.344178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.344520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.347990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:44.350770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.361558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 813/1616] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:44.361558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.369463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:44.370819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.375893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4ea85-b5ea-4a7d-a3d5-8037eaf23388 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.380983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.384700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 804/1617] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:44.384864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.384864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.384864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] PATH_INFO: `/v3/projects/6ed4aad65ded400fb55f2652eda25c0c/users/80c7bbae7fc8474c8c1cae5022dae4ed/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.385160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.402628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed4aad65ded400fb55f2652eda25c0c, user_id=80c7bbae7fc8474c8c1cae5022dae4ed, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.405336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.409786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:44.447942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f2548-c39b-48bc-818c-e8c151c77520 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.452076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.453325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.453325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.453325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.462598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.464098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.473624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 805/1618] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.474429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.476320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.488050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2151-a82b-49fa-8e34-54061f356c47 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.491758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.492470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.492824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.493206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.503480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 814/1619] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:44 2022] PUT /identity/v3/projects/6ed4aad65ded400fb55f2652eda25c0c/users/80c7bbae7fc8474c8c1cae5022dae4ed/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:44.504421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.512086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.513596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.524742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 806/1620] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.525608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.533579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f847-14cb-48ac-8e94-f3dde75bc249 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.576784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b662-98ff-4c98-82c3-ef3ee1889219 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.581563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.582220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.582503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.582779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.590066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.590928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.591403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.591833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.667536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.669842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.684165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 807/1621] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.685320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.687414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.690797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.698516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bfe3f-5041-4d58-941e-f7f3abf9bc9d service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.703963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.705154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.705590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.705985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.706109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 815/1622] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:44.706945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.708985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf32b5e-8368-4697-88df-054504096cbd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.729084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 808/1623] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:44.730067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.751147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0341b66-7180-4f68-830c-92e8019e6dfc service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.754992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.756139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.756483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.756893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] PATH_INFO: `/v3/projects/6ed4aad65ded400fb55f2652eda25c0c/users/80c7bbae7fc8474c8c1cae5022dae4ed/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.759382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed4aad65ded400fb55f2652eda25c0c, user_id=80c7bbae7fc8474c8c1cae5022dae4ed, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:44.762642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:44.775847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 816/1624] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:44 2022] PUT /identity/v3/projects/6ed4aad65ded400fb55f2652eda25c0c/users/80c7bbae7fc8474c8c1cae5022dae4ed/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:48:44.786728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.787101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.787472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:44.818916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] MFA Rules not processed for user `80c7bbae7fc8474c8c1cae5022dae4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:44.820296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:44.854137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:44.903309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 809/1625] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:44.913901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:44.946309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:44.987243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb5133-ee1f-4551-9b5b-076517ae9f1e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:44.991131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:44.991780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:44.992060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:44.992331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:45.004656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:45.005539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800227b2-e68e-4796-b8d2-c80a77504273 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:45.016450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 817/1626] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:45.266642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:45.371627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284c24-9986-4521-afa9-a20e2a92d3af None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:45.377991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:45.379470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:45.379470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:45.379470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] PATH_INFO: `/v3/users/8af59cb05c684c2d9b2df315e04ef2d0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:45.381453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af59cb05c684c2d9b2df315e04ef2d0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:45.382299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:48:45.382299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 16 12:48:45.382299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: Jul 16 12:48:45.383754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:45.669739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af59cb05c684c2d9b2df315e04ef2d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:45.716871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af59cb05c684c2d9b2df315e04ef2d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:45.721074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af59cb05c684c2d9b2df315e04ef2d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:45.721782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af59cb05c684c2d9b2df315e04ef2d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:45.737547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af59cb05c684c2d9b2df315e04ef2d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:45.817176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 810/1627] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:48:45 2022] DELETE /identity/v3/users/8af59cb05c684c2d9b2df315e04ef2d0 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:45.821183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc8d5-da33-4f1b-89dd-7b0404263bfb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:45.821668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc8d5-da33-4f1b-89dd-7b0404263bfb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:45.821668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acc8d5-da33-4f1b-89dd-7b0404263bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:45.841136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d7acc8d5-da33-4f1b-89dd-7b0404263bfb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:45.842372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d7acc8d5-da33-4f1b-89dd-7b0404263bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:45.911773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 818/1628] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:45.929031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:45.930757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:46.017808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:46.022276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966e21f-884f-4496-9baf-4b35bb8377af admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:46.028119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:46.032280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:46.032280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:46.032280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:46.049811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:46.051371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:46.065029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 811/1629] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:46.906916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:46.908362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:46.908824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:46.939786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] MFA Rules not processed for user `64a52dc9fb4a4290812e77c0c5bf8c44`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:46.941197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:47.020619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.022197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:47.038897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 819/1630] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:47.046760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bf5e8-220a-478b-89ef-20f0b09b9674 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.049757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.052750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:47.064824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.068570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.069233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.069693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] PATH_INFO: `/v3/projects/605135d537dc488db139784d0ade2446` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.073682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=605135d537dc488db139784d0ade2446)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.077203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c685f8-932e-43b2-b571-26e54dfae607 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.078840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.084147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.085280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.085938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.086319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.087222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.090913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.212734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 820/1631] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:47 2022] POST /identity/v3/projects => generated 405 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:47.217112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.256051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:47.262824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605135d537dc488db139784d0ade2446', 'request_id': 'req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.269438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605135d537dc488db139784d0ade2446', 'request_id': 'req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.300995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36909-c795-4f8b-adf9-40b24f8b11d4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.353973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605135d537dc488db139784d0ade2446', 'request_id': 'req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.370581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] The token cache is being invalidate because project 605135d537dc488db139784d0ade2446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:48:47.370857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.371621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.374330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 812/1632] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:48:47 2022] DELETE /identity/v3/projects/605135d537dc488db139784d0ade2446 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:48:47.379186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.381712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:47.418112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.419377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.419935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.653295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.653295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.653295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.656043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98c2b9-6bf2-4d03-b9fd-08e47b1b1ca5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.656043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.656043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.656043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.656769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] PATH_INFO: `/v3/users/80c7bbae7fc8474c8c1cae5022dae4ed` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.656769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c7bbae7fc8474c8c1cae5022dae4ed)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.656769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.657127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 821/1633] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:47 2022] POST /identity/v3/users => generated 309 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:47.657127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.657127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:47.657127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb25297-3a48-4a27-8baf-f2aabf43620b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.660416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.660416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.660416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.660416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.660666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.660666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.660666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 822/1634] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:48:47.660666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.660666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c4aee-129a-4c79-8e38-82ebb29cfabb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:47.660969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:47.660969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:47.660969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:47.660969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] PATH_INFO: `/v3/projects/ccc36ea8ff104f20a3158ff265f6743c/users/05b321528918447099b06728250f9413/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:47.661258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc36ea8ff104f20a3158ff265f6743c, user_id=05b321528918447099b06728250f9413, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:47.666276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:47.779187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 823/1635] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:47 2022] PUT /identity/v3/projects/ccc36ea8ff104f20a3158ff265f6743c/users/05b321528918447099b06728250f9413/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:47.783905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:47.786089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c7bbae7fc8474c8c1cae5022dae4ed'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.801397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c7bbae7fc8474c8c1cae5022dae4ed'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.802196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c7bbae7fc8474c8c1cae5022dae4ed'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.803045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c7bbae7fc8474c8c1cae5022dae4ed'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.812540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c7bbae7fc8474c8c1cae5022dae4ed'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:47.815948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:47.880985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 813/1636] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:48:47 2022] DELETE /identity/v3/users/80c7bbae7fc8474c8c1cae5022dae4ed => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:47.887809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4ea7-9564-4e02-8323-30ad87689fe8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:48.030937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:48.034402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.034402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.036083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.036792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:48.043237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0eb85d-394f-49c7-a1e1-6d6ea161b5f7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:48.069332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:48.070465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 824/1637] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:48:48.071787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:48.097326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f6807-5dad-4add-b065-c03e2eb99434 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:48.109598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:48.114452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.114452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.115287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] PATH_INFO: `/v3/projects/ccc36ea8ff104f20a3158ff265f6743c/users/05b321528918447099b06728250f9413/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.136514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc36ea8ff104f20a3158ff265f6743c, user_id=05b321528918447099b06728250f9413, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:48.144047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ebad7-783d-46d4-8bbe-30134b078a12 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:48.177824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:48.182643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 814/1638] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:48 2022] PUT /identity/v3/projects/ccc36ea8ff104f20a3158ff265f6743c/users/05b321528918447099b06728250f9413/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:48:48.277594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163de5c-82a4-43e2-a25e-0b199de85533 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.278997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163de5c-82a4-43e2-a25e-0b199de85533 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.281536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163de5c-82a4-43e2-a25e-0b199de85533 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.281763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 825/1639] 10.222.0.47 () {54 vars in 961 bytes} [Sat Jul 16 12:48:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:48:48.290952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e655af-e252-46a4-a2b4-60d40120c695 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.291711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e655af-e252-46a4-a2b4-60d40120c695 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.292026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e655af-e252-46a4-a2b4-60d40120c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.353938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b2e655af-e252-46a4-a2b4-60d40120c695 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:48.355279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b2e655af-e252-46a4-a2b4-60d40120c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:48.449950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 815/1640] 10.222.0.47 () {58 vars in 1058 bytes} [Sat Jul 16 12:48:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:48.456229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.456523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.456829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.459358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 826/1641] 10.222.0.47 () {54 vars in 912 bytes} [Sat Jul 16 12:48:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:48:48.465168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5449-3f80-41d4-87ed-619db988989c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.466011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5449-3f80-41d4-87ed-619db988989c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.466423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5449-3f80-41d4-87ed-619db988989c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.490155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-152c5449-3f80-41d4-87ed-619db988989c None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:48.491783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-152c5449-3f80-41d4-87ed-619db988989c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:48.504141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 816/1642] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:48:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:48.518129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:48.520279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:48.598331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360f441-52ba-45c7-a2ed-abd3c4603bb8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:48.605188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:48.606399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:48.606824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:48.607441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:48.620829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:48.622742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042f62b-bdba-4136-91ce-a04cc9d90119 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:48.653125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 827/1643] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:49.488765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c5449-3f80-41d4-87ed-619db988989c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:49.490964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-152c5449-3f80-41d4-87ed-619db988989c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:49.668544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c5449-3f80-41d4-87ed-619db988989c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:49.676362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:49.678561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:49.678856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:49.679137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] PATH_INFO: `/v3/projects/6ed4aad65ded400fb55f2652eda25c0c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:49.680476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed4aad65ded400fb55f2652eda25c0c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:49.682827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:49.818590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed4aad65ded400fb55f2652eda25c0c', 'request_id': 'req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:49.827540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed4aad65ded400fb55f2652eda25c0c', 'request_id': 'req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:49.835277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:49.902800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed4aad65ded400fb55f2652eda25c0c', 'request_id': 'req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:49.916382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] The token cache is being invalidate because project 6ed4aad65ded400fb55f2652eda25c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:48:49.916634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:49.917524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b4b4a0a3-4a39-4dec-a81d-c563f4896a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:48:49.919576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 817/1644] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:48:49 2022] DELETE /identity/v3/projects/6ed4aad65ded400fb55f2652eda25c0c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:49.924892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:49.926427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:49.926923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:49.949125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:49.950597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:50.018238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 828/1645] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:50.026470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.026981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.027364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.051484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:50.052845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:50.065080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 818/1646] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:50.067319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.089762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8af08c-186d-44f0-bf5a-7bb3fe92f820 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.105219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.105989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.106244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.106522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.108509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:48:50.108832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.111180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.139013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 829/1647] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:48:50.143648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.153899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3215b2-b5ed-4c93-8cf1-d82fb3b8c354 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.161314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.162682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.162977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.163760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.165402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.167947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.279028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 819/1648] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:50 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:50.280687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.294324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0bf0b-6609-4206-8368-011edab73f9b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.367651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.368377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.368666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.368948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.369688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.371824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.380519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:50.454763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 830/1649] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:50 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:50.456993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.469366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887720a9-df68-4b95-9109-2c8d51e8feb3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.478192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.478890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.479172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.481632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.481632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.481912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.496930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 820/1650] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:50.498007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.508572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed5ce-6ea6-4a70-99d4-232c0a9b424a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.518413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.519102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.519386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.519664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.520403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.522751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a449-e50c-47b0-8a6d-7b24dd76fb1f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.549418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 831/1651] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:48:50.550898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:50.568712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a765e-c5c9-42ae-bc62-ee6157c39cb7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:50.585691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:50.586615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.586898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.587176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] PATH_INFO: `/v3/projects/998c91a1e94b4ccea621f16e222ff115/users/c44311214b6b46e3a9d4aa4ec9b82f9f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.632847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=998c91a1e94b4ccea621f16e222ff115, user_id=c44311214b6b46e3a9d4aa4ec9b82f9f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:50.636434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:50.733565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 821/1652] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:50 2022] PUT /identity/v3/projects/998c91a1e94b4ccea621f16e222ff115/users/c44311214b6b46e3a9d4aa4ec9b82f9f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:50.746078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb8ed-f5ff-4e5e-b90e-ed3a6388bee8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:50.746370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb8ed-f5ff-4e5e-b90e-ed3a6388bee8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:50.746646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cb8ed-f5ff-4e5e-b90e-ed3a6388bee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:50.815041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e52cb8ed-f5ff-4e5e-b90e-ed3a6388bee8 None None] MFA Rules not processed for user `c44311214b6b46e3a9d4aa4ec9b82f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:50.816286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e52cb8ed-f5ff-4e5e-b90e-ed3a6388bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:50.966870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 832/1653] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:50.968144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:51.157076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d032-7cbf-485e-aa6e-43ccfb128314 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:51.193367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:51.194068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.194355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.194629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.195353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:51.197338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f662a-abdc-462e-8277-f9f875a96959 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:51.367949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b849-6956-418b-926a-f38673803bec None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.367949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b849-6956-418b-926a-f38673803bec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.367949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b849-6956-418b-926a-f38673803bec None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.369470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 833/1654] 10.222.0.47 () {54 vars in 961 bytes} [Sat Jul 16 12:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:48:51.383153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 822/1655] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:48:50 2022] POST /identity/v3/projects => generated 381 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:51.384414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b849-6956-418b-926a-f38673803bec None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:51.424764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681733-e3f2-4d02-aeb5-d42c8b2707dc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.425079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681733-e3f2-4d02-aeb5-d42c8b2707dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.428172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681733-e3f2-4d02-aeb5-d42c8b2707dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.459217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f2681733-e3f2-4d02-aeb5-d42c8b2707dc None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:51.465991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f2681733-e3f2-4d02-aeb5-d42c8b2707dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:51.651180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:51.679602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b849-6956-418b-926a-f38673803bec None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:51.696701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:51.697123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.697416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.697689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.705439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:51.708449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:51.711817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 823/1656] 10.222.0.47 () {58 vars in 1058 bytes} [Sat Jul 16 12:48:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:51.726277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21219929-8865-4ace-9939-7081d9940a20 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.726713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21219929-8865-4ace-9939-7081d9940a20 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.726996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21219929-8865-4ace-9939-7081d9940a20 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.749075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 824/1657] 10.222.0.47 () {54 vars in 912 bytes} [Sat Jul 16 12:48:51 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 12:48:51.780511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.780788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.781109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.857738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 834/1658] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:48:51 2022] POST /identity/v3/users => generated 297 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:48:51.861488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:51.867955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:51.869715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:51.889445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff2ae-fdc9-46f2-871f-f187ea0edcd0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:51.890311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 825/1659] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:48:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:51.910844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:51.912413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:51.915837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:51.917401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:51.919059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:51.919793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:51.921689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:51.931428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:51.981710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 835/1660] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:51.990392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:52.023037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb2f23-7ba5-47df-a4ae-b189536aa2da admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:52.029665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:52.033506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:52.034087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:52.034609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] PATH_INFO: `/v3/projects/c25e0ef62eb74f728551f722f3cc548b/users/0eaf294e8a1e4d73ae59e47b2202975c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:52.056121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e0ef62eb74f728551f722f3cc548b, user_id=0eaf294e8a1e4d73ae59e47b2202975c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:52.060105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:52.094850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dd81e-4649-4e93-aaee-9bdbb6a9c6c1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:52.099662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:52.100526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:52.105553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:52.105553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:52.114878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:52.118364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:52.140315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 826/1661] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:52.191052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 836/1662] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:51 2022] PUT /identity/v3/projects/c25e0ef62eb74f728551f722f3cc548b/users/0eaf294e8a1e4d73ae59e47b2202975c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:52.195334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:52.210926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e39c9-670b-4701-9299-d19f4c0bcc4a service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:52.382359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:52.386654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:52.387542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:52.387833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:52.389150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:52.393472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df01e6a-279a-4f6d-bb66-89d424fc9bc7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:52.422327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 827/1663] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:48:52.424234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:52.443773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccaeee-0555-4419-877b-538316917d4f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:52.458124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:52.459212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:52.464079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:52.464079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] PATH_INFO: `/v3/projects/c25e0ef62eb74f728551f722f3cc548b/users/0eaf294e8a1e4d73ae59e47b2202975c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:52.472674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e0ef62eb74f728551f722f3cc548b, user_id=0eaf294e8a1e4d73ae59e47b2202975c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:52.478561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:52.526801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 837/1664] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:48:52 2022] PUT /identity/v3/projects/c25e0ef62eb74f728551f722f3cc548b/users/0eaf294e8a1e4d73ae59e47b2202975c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:48:52.537941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:52.538227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:52.546903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:52.623090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] MFA Rules not processed for user `0eaf294e8a1e4d73ae59e47b2202975c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:52.630101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:52.872388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 828/1665] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:52.884856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:52.916912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:52.997578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb1d16-2c28-4b1f-912e-a890e1db9b05 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:53.005393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:53.007126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:53.007868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:53.008368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:53.038888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:53.042576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1903795-c2da-4d77-a7de-a50b9d2864fa service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:53.107072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 838/1666] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:53.440158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:53.458677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af61df-9ca0-4a27-8830-0e51e0c372be None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:53.461846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:53.462618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:53.463013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:53.463801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:53.478982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:53.566156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:53.568818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad9c57-2b03-4434-88c2-422192a6be0c service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:53.586984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 829/1667] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:48:54.296035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd5d3-e8e0-4474-8ba9-e6d5055a4310 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:54.296570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd5d3-e8e0-4474-8ba9-e6d5055a4310 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:54.296570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd5d3-e8e0-4474-8ba9-e6d5055a4310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:54.400772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-58efd5d3-e8e0-4474-8ba9-e6d5055a4310 None None] MFA Rules not processed for user `05b321528918447099b06728250f9413`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:54.406118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-58efd5d3-e8e0-4474-8ba9-e6d5055a4310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:54.658916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 839/1668] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:57.443093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6ce-cb79-47ae-9cc2-71885c71bd2b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:57.443686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6ce-cb79-47ae-9cc2-71885c71bd2b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:57.443686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6ce-cb79-47ae-9cc2-71885c71bd2b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:57.445499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 830/1669] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:48:57.455444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349efa88-48d2-42d4-a433-04b1664811a3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:57.456312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349efa88-48d2-42d4-a433-04b1664811a3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:57.456734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349efa88-48d2-42d4-a433-04b1664811a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:57.500081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-349efa88-48d2-42d4-a433-04b1664811a3 None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:57.502111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-349efa88-48d2-42d4-a433-04b1664811a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:57.677693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 840/1670] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:48:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:48:57.695334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9386-9f0d-46c9-84bb-33ab2465cb64 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:57.695939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9386-9f0d-46c9-84bb-33ab2465cb64 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:57.696385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb9386-9f0d-46c9-84bb-33ab2465cb64 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:57.698346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 831/1671] 10.222.0.47 () {54 vars in 960 bytes} [Sat Jul 16 12:48:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:48:57.705330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3274e-b9b2-4265-bd32-001e2efaec26 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:57.705922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3274e-b9b2-4265-bd32-001e2efaec26 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:57.706403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3274e-b9b2-4265-bd32-001e2efaec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:57.761216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c6f3274e-b9b2-4265-bd32-001e2efaec26 None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:57.762590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c6f3274e-b9b2-4265-bd32-001e2efaec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:57.801543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb9386-9f0d-46c9-84bb-33ab2465cb64 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:57.806272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 841/1672] 10.222.0.47 () {58 vars in 1057 bytes} [Sat Jul 16 12:48:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:48:57.834716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb9386-9f0d-46c9-84bb-33ab2465cb64 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:57.845656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:57.847956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:57.848428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:57.849599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:57.869753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:57.872442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:57.899679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 832/1673] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:48:58.951151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9446d-0a96-4336-8250-078816b17bdc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:58.951645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9446d-0a96-4336-8250-078816b17bdc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:58.951877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9446d-0a96-4336-8250-078816b17bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:59.069673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-45a9446d-0a96-4336-8250-078816b17bdc None None] MFA Rules not processed for user `3e12d902f94b40cea088dc6f24dcb444`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:48:59.072773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-45a9446d-0a96-4336-8250-078816b17bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:48:59.136438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:48:59.235437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 842/1674] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:48:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:48:59.264407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:48:59.458292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ee36-0532-470d-9588-2f0f1282d89d service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:48:59.482949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:48:59.483931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:48:59.484366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:48:59.484793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:48:59.509953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:48:59.520028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e7a6f-7402-4281-b621-0a347af83877 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:48:59.553782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 833/1675] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:48:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:00.859398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca84f-8920-4094-924c-70c43186dc85 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:00.861163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca84f-8920-4094-924c-70c43186dc85 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:00.861725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca84f-8920-4094-924c-70c43186dc85 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:00.863285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 843/1676] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:49:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:00.881592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:00.881592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:00.881592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:00.927551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] MFA Rules not processed for user `c73b6a617d4546599d336eeb5b45e858`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:00.929498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:00.952845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 834/1677] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:49:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:01.099541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ca84f-8920-4094-924c-70c43186dc85 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:01.132156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ca84f-8920-4094-924c-70c43186dc85 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:01.150658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:01.151694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:01.156866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:01.156866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:01.178828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:01.181777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576198c-5dad-4e81-b014-d75a529c851f service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:01.220869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 844/1678] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:01.294426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:01.296016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:01.405758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318ee5c-520f-48ba-966d-ae14ef47651b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:01.415317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:01.417985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:01.418675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:01.420025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] PATH_INFO: `/v3/users/64a52dc9fb4a4290812e77c0c5bf8c44` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:01.422044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a52dc9fb4a4290812e77c0c5bf8c44)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:01.427468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:01.466430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:01.668676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a52dc9fb4a4290812e77c0c5bf8c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:01.722291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a52dc9fb4a4290812e77c0c5bf8c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:01.723946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a52dc9fb4a4290812e77c0c5bf8c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:01.724854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a52dc9fb4a4290812e77c0c5bf8c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:01.742458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e38d04d-da9c-44cd-8fc5-5f6e450b214f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a52dc9fb4a4290812e77c0c5bf8c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:01.858056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 835/1679] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:01 2022] DELETE /identity/v3/users/64a52dc9fb4a4290812e77c0c5bf8c44 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:02.923056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:02.923539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:02.923539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:02.987160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] MFA Rules not processed for user `f1c789055f344bb59a86b6d4fe5cac91`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:02.990317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:02.994319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8702ca7-e93e-48da-97a8-28ecfeb8ad2e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:02.994319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8702ca7-e93e-48da-97a8-28ecfeb8ad2e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:02.994319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8702ca7-e93e-48da-97a8-28ecfeb8ad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:03.071513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f8702ca7-e93e-48da-97a8-28ecfeb8ad2e None None] MFA Rules not processed for user `46b5c80d8bf34965a30b9b67e9117996`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:03.072855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f8702ca7-e93e-48da-97a8-28ecfeb8ad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:03.202123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 845/1680] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:03.204303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:03.207883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:03.274334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 836/1681] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:03.276053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:03.281468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:03.281468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:03.340935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] MFA Rules not processed for user `f1c789055f344bb59a86b6d4fe5cac91`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:03.353425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:03.381995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 837/1682] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:03.383580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:03.387806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:03.411340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd9a7-c5d2-40d3-8bc4-5f3b01f98014 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:03.445482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:03.446840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:03.447679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:03.448293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] PATH_INFO: `/v3/projects/3b76170e55944f5786c114512c695204` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:03.451315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b76170e55944f5786c114512c695204)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:03.452785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 12:49:03.452785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 16 12:49:03.452785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: Jul 16 12:49:03.456098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:03.621439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:03.656246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b76170e55944f5786c114512c695204', 'request_id': 'req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:03.675782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9089b-f02c-4e2e-aa6d-99e1bfd38275 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:03.691813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b76170e55944f5786c114512c695204', 'request_id': 'req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:03.722326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:03.783304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b76170e55944f5786c114512c695204', 'request_id': 'req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:03.810068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] The token cache is being invalidate because project 3b76170e55944f5786c114512c695204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:03.810711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:03.815666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:03.830968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 846/1683] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:03 2022] DELETE /identity/v3/projects/3b76170e55944f5786c114512c695204 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:03.832861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:03.835267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:03.852512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:03.853207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:03.853491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:03.853766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:03.856720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.007990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:04.010246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc84646-95c9-4a51-933e-75c9caf2cac6 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:04.036235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 838/1684] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:04.040796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be0788-ab2f-4c0e-ad1c-7503b35fb76d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:04.042979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.044287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.044711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:04.056422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:04.057356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.057899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.058181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:04.064066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.136329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] MFA Rules not processed for user `628e662a3b49438f9930a2d160658b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:04.137583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.237103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:04.240780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:04.267753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 847/1685] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:04.268745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:04.272565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.442098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 839/1686] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:04.452369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:04.454433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.495950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7c6b-1f79-46de-ba79-b7bd35a0b413 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:04.508289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30682b2-ad81-4f7f-a346-b86ab81ba53a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:04.513164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:04.514151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.514426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.514717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] PATH_INFO: `/v3/users/05b321528918447099b06728250f9413` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:04.517304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b321528918447099b06728250f9413)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:04.521503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:04.527990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:04.535127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.535352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:04.535438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.535438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:04.553059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:04.555894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c641b6-9199-4eb8-9ae3-379ae19e8cff service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:04.582722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 840/1687] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:04.757524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b321528918447099b06728250f9413'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:04.776633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.776923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.777234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:04.786651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b321528918447099b06728250f9413'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:04.825551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b321528918447099b06728250f9413'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:04.826420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b321528918447099b06728250f9413'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:04.842412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] MFA Rules not processed for user `f1c9a7f942c541a287da38a894f1d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:04.867266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:04.895627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:04.919097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1fffefa8-720b-4e16-88dc-34a21a5335d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b321528918447099b06728250f9413'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:04.927049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 848/1688] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:04 2022] DELETE /identity/v3/users/05b321528918447099b06728250f9413 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:04.968044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 841/1689] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:04.969902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:04.970190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:04.970473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:05.010401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] MFA Rules not processed for user `f1c9a7f942c541a287da38a894f1d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:05.011623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:05.164494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 849/1690] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:05.181386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:05.185223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:05.217632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaaf8e-eb78-4785-860a-7d7ce4b4c8cf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:05.343044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:05.344552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:05.399475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:05.399475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:05.399475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:05.399475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:05.422448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:05.433406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf9e37-b776-42ad-af18-1799d763a87e service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:05.448897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 842/1691] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:05.542321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17ff2c-3901-4934-9a6c-1afce4763397 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:05.552152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:05.554424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:05.555535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:05.555824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] PATH_INFO: `/v3/projects/ccc36ea8ff104f20a3158ff265f6743c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:05.558826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc36ea8ff104f20a3158ff265f6743c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:05.561825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:05.771794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:05.803263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc36ea8ff104f20a3158ff265f6743c', 'request_id': 'req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:05.819103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc36ea8ff104f20a3158ff265f6743c', 'request_id': 'req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:05.888977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ff53-fcdd-48dd-abea-96c575a63936 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:05.889352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ff53-fcdd-48dd-abea-96c575a63936 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:05.889663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ff53-fcdd-48dd-abea-96c575a63936 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:05.891680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 843/1692] 10.222.0.47 () {54 vars in 912 bytes} [Sat Jul 16 12:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:05.899991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139194-f7e8-47b2-a246-b1fe74e3e917 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:05.900290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139194-f7e8-47b2-a246-b1fe74e3e917 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:05.900556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139194-f7e8-47b2-a246-b1fe74e3e917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:05.924734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc36ea8ff104f20a3158ff265f6743c', 'request_id': 'req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:05.953382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] The token cache is being invalidate because project ccc36ea8ff104f20a3158ff265f6743c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:05.959655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:05.959655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:05.977438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 850/1693] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:05 2022] DELETE /identity/v3/projects/ccc36ea8ff104f20a3158ff265f6743c => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:05.978428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:05.982649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.031165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b0139194-f7e8-47b2-a246-b1fe74e3e917 None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:06.039187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b0139194-f7e8-47b2-a246-b1fe74e3e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.197616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b682c3e-dcbb-40e0-9db1-40ab0a9e6849 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:06.211602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:06.214457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.214892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.215278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 844/1694] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:49:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:06.217104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.217558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] PATH_INFO: `/v3/users/09eca28965ab4ddfbe614704e8e884d6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.218646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.220804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.222596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09eca28965ab4ddfbe614704e8e884d6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:06.226452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:06.266787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:06.273253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.299463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 845/1695] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:06.299463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:06.299463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.349359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901afe9-5564-4e3f-9b67-9f5fbf1a3828 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:06.354152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:06.355545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.356027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.356824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.371126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:06.373522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72300f19-bd7e-442b-8b96-addfcb4c3353 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:06.387326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 846/1696] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:06.388897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80541-01d8-4b79-95ad-2a18baccb96a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.389446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80541-01d8-4b79-95ad-2a18baccb96a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.389944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b80541-01d8-4b79-95ad-2a18baccb96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.434851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-83b80541-01d8-4b79-95ad-2a18baccb96a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:06.439601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-83b80541-01d8-4b79-95ad-2a18baccb96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.477462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 847/1697] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:06.479460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43297-15c9-44e3-b204-59414e1a53cb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.497981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43297-15c9-44e3-b204-59414e1a53cb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.498262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43297-15c9-44e3-b204-59414e1a53cb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.502237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09eca28965ab4ddfbe614704e8e884d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:06.503626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 848/1698] 10.222.0.47 () {54 vars in 912 bytes} [Sat Jul 16 12:49:06 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:06.504492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43297-15c9-44e3-b204-59414e1a53cb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:06.525258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09eca28965ab4ddfbe614704e8e884d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:06.544746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43297-15c9-44e3-b204-59414e1a53cb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:06.553415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:06.554473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.554830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.555308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.556034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09eca28965ab4ddfbe614704e8e884d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:06.556339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:06.556831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:06.559124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c427944-bfd7-4bcb-968b-60fe159b05c1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:06.559582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09eca28965ab4ddfbe614704e8e884d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:06.575930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 849/1699] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:06.577418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.577705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.577979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.586576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:06.629202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:06.630492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.650127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 850/1700] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:06.653629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:06.668513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd431427-2f84-4f2e-a6c9-fe91fd355f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09eca28965ab4ddfbe614704e8e884d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:06.679871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a0c4-02ec-4d1d-be2e-1cb160ef094c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:06.686234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:06.687005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.687363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.688357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.689237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:06.691472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 851/1701] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:05 2022] DELETE /identity/v3/users/09eca28965ab4ddfbe614704e8e884d6 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:06.692113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:06.696726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:06.698488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:06.698969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:06.732151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:06.733780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.827713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 851/1702] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:06.828960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:06.839037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:06.987834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 852/1703] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:06.993434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.003861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:07.074777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:07.076226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29014e00-f63c-4b53-8727-6632bfefdff3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.110073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc249f80-c0c2-444c-8e46-7e4119d935ec admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.129765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.130447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.130736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.131014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.143538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:07.173864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.173864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.197705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 852/1704] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:07.198675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.208307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.211184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.211644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.216799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.216799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.216799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2421f4-75b3-49b8-9838-01f8ebc9a9b9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.235895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8878d-786a-4672-9897-61a072c7a6e0 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.252750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.254490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.256928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.257404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.269110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.270959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.291206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 853/1705] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:07.349069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 853/1706] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:06 2022] POST /identity/v3/users => generated 291 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:07.353526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.377953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2363-216f-4b12-888d-243e5b937f60 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.383917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.384910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.385854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.386975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.387903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.390492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.401198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.401559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.401848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.457720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 854/1707] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:07.462315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.463707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] MFA Rules not processed for user `3e12d902f94b40cea088dc6f24dcb444`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:07.466163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:07.480499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fecb06-10b8-48e9-99f6-26a2a015bb3b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.499933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.501353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.501689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.501941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.508776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.513112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.530943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 855/1708] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:07.535484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.606798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab738e2-687f-48dd-bae1-c19ac2ea086c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.626686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:07.632802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.633152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.633439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] PATH_INFO: `/v3/projects/64812b0063ce47fc9dfa84a094b448e4/users/f25a8a9d2d5a453f820aad105c31dd1a/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.683457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64812b0063ce47fc9dfa84a094b448e4, user_id=f25a8a9d2d5a453f820aad105c31dd1a, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:07.691898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 854/1709] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:07.692954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:07.697863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e842d-42eb-4e33-8602-73fac6110b15 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:07.700725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:07.790128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 856/1710] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:07 2022] PUT /identity/v3/projects/64812b0063ce47fc9dfa84a094b448e4/users/f25a8a9d2d5a453f820aad105c31dd1a/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:07.791946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:07.793341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:07.794189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:07.878450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] MFA Rules not processed for user `f25a8a9d2d5a453f820aad105c31dd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:07.881382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:07.889219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6af12-543c-4f0c-8d8c-a4ebc23bc03d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:07.915470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:08.105636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 857/1711] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:08.105820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.108173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:08.109940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.110249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.114330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:08.119114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:08.130593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:08.163076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:08.164711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:08.191880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 855/1712] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:08.286070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3162eec-8256-4a60-a5b3-ffe1ed7cc6a2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:08.309900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.310602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.310888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.311164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:08.311921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:08.315054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:08.517362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 858/1713] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:08 2022] POST /identity/v3/projects => generated 385 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:08.521306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:08.534507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cc444-56fc-4fa0-adab-03b8e5189f10 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:08.625624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.626331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.626698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.627008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:08.627723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:08.642764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:08.759742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 856/1714] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:08 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:08.764566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:08.783159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301b005-8782-4c05-b01e-9e7c461be5d4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:08.797325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.798088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.798374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.798664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:08.799407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:08.802039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:08.850057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 859/1715] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:08.853665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:08.876512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa4dd1-94e9-4d93-90ba-f2c37611c176 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:08.892055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.892055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.892248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.892451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:08.893296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:08.894870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aea32c-dd1a-400e-bfe8-c76ee4193630 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:08.920698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 857/1716] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:08.925747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:08.944132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442568a-af63-4c27-bfdd-60bef1356489 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:08.976864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:08.986319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:08.988298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:08.988298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] PATH_INFO: `/v3/projects/dc1516186d404516921dc4c047a14e10/users/515dccd9e4544808b754d24ee2da9472/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:09.010785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1516186d404516921dc4c047a14e10, user_id=515dccd9e4544808b754d24ee2da9472, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:09.014016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:09.122055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 860/1717] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:08 2022] PUT /identity/v3/projects/dc1516186d404516921dc4c047a14e10/users/515dccd9e4544808b754d24ee2da9472/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:09.138821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895ce4-7378-4e6a-90cb-d76163ea5202 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:09.139129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895ce4-7378-4e6a-90cb-d76163ea5202 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:09.139442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895ce4-7378-4e6a-90cb-d76163ea5202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:09.197265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d0895ce4-7378-4e6a-90cb-d76163ea5202 None None] MFA Rules not processed for user `515dccd9e4544808b754d24ee2da9472`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:09.199333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d0895ce4-7378-4e6a-90cb-d76163ea5202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:09.362320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 858/1718] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:09.367020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:09.548798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b09c1-8f95-4a4d-b03c-5d182d10cf48 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:09.563463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:09.566676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:09.566676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:09.566676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:09.566676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:09.568791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:09.675025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:09.675586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:09.675586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:09.710257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] MFA Rules not processed for user `f1c9a7f942c541a287da38a894f1d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:09.711690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:09.718537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 861/1719] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:09 2022] POST /identity/v3/projects => generated 387 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:09.722501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:09.728647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:09.736400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:09.902050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 859/1720] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:09.903362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:09.948241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac1792-a4b6-40b8-9361-baf878a5998e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:09.999700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:10.081292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f31f9-ac2d-4419-92d0-ef00a41c213b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:10.111728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:10.114312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.114889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.115368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.115937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:10.116840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.119014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:10.120078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.120375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] PATH_INFO: `/v3/projects/aa39d77033fe427296b7e5fa6a49f3d3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.124031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:10.124999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa39d77033fe427296b7e5fa6a49f3d3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:10.135556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:10.226870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 860/1721] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:09 2022] POST /identity/v3/users => generated 300 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:10.229071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:10.262503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c0a4-ca02-43da-9068-feb893d0e19a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:10.278765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:10.280474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.280971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.282828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.287201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:10.291621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:10.328621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 861/1722] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:10.333572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:10.357120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54339f2a-b2af-4e75-bd7a-01f26cb33725 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:10.371393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:10.372480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.372905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.373359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] PATH_INFO: `/v3/projects/a1ccd2da06a543fe813b99c70840338a/users/63a1a6c295974f43bb9ae8ce3ae220df/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.394411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa39d77033fe427296b7e5fa6a49f3d3', 'request_id': 'req-d48da345-8893-4acf-a235-066b516828b9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:10.413726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa39d77033fe427296b7e5fa6a49f3d3', 'request_id': 'req-d48da345-8893-4acf-a235-066b516828b9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:10.449770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ccd2da06a543fe813b99c70840338a, user_id=63a1a6c295974f43bb9ae8ce3ae220df, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:10.464796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:10.489848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa39d77033fe427296b7e5fa6a49f3d3', 'request_id': 'req-d48da345-8893-4acf-a235-066b516828b9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:10.506957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] The token cache is being invalidate because project aa39d77033fe427296b7e5fa6a49f3d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:10.507257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:10.509987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d48da345-8893-4acf-a235-066b516828b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:10.514412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 862/1723] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:09 2022] DELETE /identity/v3/projects/aa39d77033fe427296b7e5fa6a49f3d3 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:10.520340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c5d90-b50e-4a0a-875e-2bee174469e1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.520824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c5d90-b50e-4a0a-875e-2bee174469e1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.526527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c5d90-b50e-4a0a-875e-2bee174469e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.574223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-912c5d90-b50e-4a0a-875e-2bee174469e1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:10.575753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-912c5d90-b50e-4a0a-875e-2bee174469e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:10.600584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 862/1724] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:10 2022] PUT /identity/v3/projects/a1ccd2da06a543fe813b99c70840338a/users/63a1a6c295974f43bb9ae8ce3ae220df/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:10.604512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:10.607135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:10.847112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 863/1725] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:10.854369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.854796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.854991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.898513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd93e-ffae-4bed-bff6-09a6dea84342 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:10.913300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:10.914825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:10.915821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:10.916276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:10.917325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:10.919678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:10.922314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:10.930499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:10.961685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 864/1726] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:10.965984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:10.981998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 863/1727] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:10.983215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:11.016253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d1bc-5a6f-4155-841f-91ec4b5f5f58 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:11.022569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863042e1-23d9-4467-906c-b59bcd37e142 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:11.030978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:11.032445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.035067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.035067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:11.035224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:11.035309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:11.035309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:11.035446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.035446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.035446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] PATH_INFO: `/v3/projects/a1ccd2da06a543fe813b99c70840338a/users/63a1a6c295974f43bb9ae8ce3ae220df/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:11.040000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ccd2da06a543fe813b99c70840338a, user_id=63a1a6c295974f43bb9ae8ce3ae220df, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:11.044105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:11.049672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:11.069058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:11.096129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 865/1728] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:11.096129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:11.106346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:11.107039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 864/1729] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:10 2022] PUT /identity/v3/projects/a1ccd2da06a543fe813b99c70840338a/users/63a1a6c295974f43bb9ae8ce3ae220df/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:11.122181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:11.138863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498ca1-3083-4c57-96a6-fc94a5a8b19a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:11.313395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:11.339353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:11.340275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.340564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.340844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] PATH_INFO: `/v3/users/c44311214b6b46e3a9d4aa4ec9b82f9f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:11.342849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44311214b6b46e3a9d4aa4ec9b82f9f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:11.346926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:11.384414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a90ce1-d5a6-4364-8ca7-f234343969d8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:11.395928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:11.400499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.400949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.401893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:11.403245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:11.407290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaaae1-c393-4eee-9586-b2f670a47d0a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:11.633848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 865/1730] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:11 2022] POST /identity/v3/projects => generated 363 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:11.640444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-7bc4-482a-930c-c869dab57573 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.640444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-7bc4-482a-930c-c869dab57573 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.640444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7152f0-7bc4-482a-930c-c869dab57573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:11.701761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8b7152f0-7bc4-482a-930c-c869dab57573 None None] MFA Rules not processed for user `63a1a6c295974f43bb9ae8ce3ae220df`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:11.704070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8b7152f0-7bc4-482a-930c-c869dab57573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:11.745889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44311214b6b46e3a9d4aa4ec9b82f9f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:11.829247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44311214b6b46e3a9d4aa4ec9b82f9f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:11.837184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44311214b6b46e3a9d4aa4ec9b82f9f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:11.838086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44311214b6b46e3a9d4aa4ec9b82f9f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:11.866657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44311214b6b46e3a9d4aa4ec9b82f9f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:11.964927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 866/1731] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:11 2022] DELETE /identity/v3/users/c44311214b6b46e3a9d4aa4ec9b82f9f => generated 0 bytes in 872 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:11.966081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:11.969881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:11.978356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 866/1732] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:11.979877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:11.980202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:11.981295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.033789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] MFA Rules not processed for user `628e662a3b49438f9930a2d160658b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:12.039930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:12.117606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4719d5-5f81-459e-abdc-f94cc9055440 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.138585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.139538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.144290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.144984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.148353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.150764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318d26d-4d0a-4b00-9afd-19dc169ebe25 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.293389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 867/1733] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:11 2022] POST /identity/v3/users => generated 282 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:12.295431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.299377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.299658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.320408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 867/1734] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:12.321471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:12.332975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:12.357949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:12.374242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:12.378232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:12.398160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 868/1735] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:12.399195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:12.431509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca95061-31c8-411d-be96-bca81e1d7bd6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.470147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.473075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.473453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.476928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397e35f-46c5-4fdb-b47a-7a4c828c823e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.477446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.478697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.481517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.483499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.487771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.487771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.487771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.509397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 869/1736] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:12.514365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:12.521231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:12.575663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.577663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.597582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 868/1737] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:12.598463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:12.611847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0702e-0d87-422f-b33a-c08f96c86ca7 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.618424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.619081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.619846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.619846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.621304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.623848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.641188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 869/1738] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:12.642727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:12.659519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1f2b-cb78-48ff-87dd-0003bdbf462e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.677326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.678232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.678895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.678895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] PATH_INFO: `/v3/projects/99aeabea578d48a6b8bd6107154b0a8d/users/7f16a5fd7b814a5aa700c31e82a7c705/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.732342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225fc26-a8c7-4e2b-8509-d5287537b168 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:12.748347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99aeabea578d48a6b8bd6107154b0a8d, user_id=7f16a5fd7b814a5aa700c31e82a7c705, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.753162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:12.760915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:12.761293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.761678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.762458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.762729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:12.787294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:12.790330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a56adb-f285-4f18-b50c-933101c97cb2 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:12.826306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 870/1739] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:12.920334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 870/1740] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:12 2022] PUT /identity/v3/projects/99aeabea578d48a6b8bd6107154b0a8d/users/7f16a5fd7b814a5aa700c31e82a7c705/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:12.933544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b0d-00b6-4d6a-a2ad-f5f0e1c6506d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:12.934753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b0d-00b6-4d6a-a2ad-f5f0e1c6506d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:12.934917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b0d-00b6-4d6a-a2ad-f5f0e1c6506d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:13.044917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b9f08b0d-00b6-4d6a-a2ad-f5f0e1c6506d None None] MFA Rules not processed for user `7f16a5fd7b814a5aa700c31e82a7c705`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:13.046269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b9f08b0d-00b6-4d6a-a2ad-f5f0e1c6506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:13.251816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 871/1741] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:13.256118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:13.468680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:13.469288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:13.469865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:13.486817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce67b4c-7ba8-4912-bf31-ae1e3ec7025c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:13.503575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:13.504511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:13.504939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:13.505374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:13.506392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:13.512565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] MFA Rules not processed for user `46b5c80d8bf34965a30b9b67e9117996`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:13.513291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:13.514928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:13.629756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 872/1742] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:13.644506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 871/1743] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:13 2022] POST /identity/v3/projects => generated 363 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:13.649588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:13.673432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0be8d-6012-4de1-a4cd-1dfc601083b1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:13.781138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:13.781723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:13.781976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:13.782824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:13.783538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:13.785886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:13.798718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:13.801105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:13.838628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fbefe-683f-435f-b74f-bcc6997a1416 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:13.869642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:13.870568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:13.881792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:13.882109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] PATH_INFO: `/v3/projects/998c91a1e94b4ccea621f16e222ff115` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:13.883814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=998c91a1e94b4ccea621f16e222ff115)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:13.888752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:13.895260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:13.987268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 873/1744] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:13 2022] POST /identity/v3/users => generated 288 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:13.989469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:14.020219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5220d17-bbf6-471c-a726-c8dee8f7b7ed admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:14.042854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:14.043846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:14.044357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:14.044782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:14.045763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:14.048155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:14.074575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 874/1745] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:14.089626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:14.108277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c94c0e-d398-456e-8af6-df855b60dac2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:14.147329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998c91a1e94b4ccea621f16e222ff115', 'request_id': 'req-70602b99-4674-4c4a-8f29-8613aba49800'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.176923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:14.185903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998c91a1e94b4ccea621f16e222ff115', 'request_id': 'req-70602b99-4674-4c4a-8f29-8613aba49800'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.195843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:14.208576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:14.210186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:14.210702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] PATH_INFO: `/v3/projects/7e890c573f7d446d864314719b1b69a4/users/7bd83a97735e4cd1acaebf21f050c334/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:14.248704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e890c573f7d446d864314719b1b69a4, user_id=7bd83a97735e4cd1acaebf21f050c334, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:14.252355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:14.289434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998c91a1e94b4ccea621f16e222ff115', 'request_id': 'req-70602b99-4674-4c4a-8f29-8613aba49800'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.306880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] The token cache is being invalidate because project 998c91a1e94b4ccea621f16e222ff115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:14.319616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.320591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.335685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 872/1746] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:13 2022] DELETE /identity/v3/projects/998c91a1e94b4ccea621f16e222ff115 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:14.339279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:14.342135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:14.381168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602b99-4674-4c4a-8f29-8613aba49800 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:14.387762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 875/1747] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:14 2022] PUT /identity/v3/projects/7e890c573f7d446d864314719b1b69a4/users/7bd83a97735e4cd1acaebf21f050c334/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:14.392524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:14.394599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:14.528744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3eaa9-2ead-437b-8f1f-512ef208b9b2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:14.594767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:14.595835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:14.596122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:14.596420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:14.601618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:14.603559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:14.659742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 876/1748] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:14.661974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:14.667738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:14.688414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:14.690183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:14.690884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:14.691263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] PATH_INFO: `/v3/users/0eaf294e8a1e4d73ae59e47b2202975c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:14.693798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eaf294e8a1e4d73ae59e47b2202975c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:14.710974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:14.895961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842474ec-58af-4996-be6d-e47daafc9c8b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:14.912691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:14.920887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:14.921990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:14.922565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:14.923570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eaf294e8a1e4d73ae59e47b2202975c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.925420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:14.934655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eaf294e8a1e4d73ae59e47b2202975c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:14.999343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eaf294e8a1e4d73ae59e47b2202975c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:15.003423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eaf294e8a1e4d73ae59e47b2202975c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:15.122396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:15.123866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fb92d-36fb-4b92-888d-d67216eaaed1 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:15.130515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eaf294e8a1e4d73ae59e47b2202975c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:15.146755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 873/1749] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:14 2022] DELETE /identity/v3/users/0eaf294e8a1e4d73ae59e47b2202975c => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:15.147847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:15.149886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:15.150076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 877/1750] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:15.151624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:15.152073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:15.152533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:15.289434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:15.290916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:15.403053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:15.410237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07348-4802-4b47-9219-022d8595e522 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:15.424451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:15.430621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:15.430621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:15.430621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] PATH_INFO: `/v3/projects/7e890c573f7d446d864314719b1b69a4/users/7bd83a97735e4cd1acaebf21f050c334/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:15.437469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e890c573f7d446d864314719b1b69a4, user_id=7bd83a97735e4cd1acaebf21f050c334, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:15.444176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4263ba-7d32-42e8-9f7c-2a7136100b8f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:15.494568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 878/1751] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:15.497884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 874/1752] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:15 2022] PUT /identity/v3/projects/7e890c573f7d446d864314719b1b69a4/users/7bd83a97735e4cd1acaebf21f050c334/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:15.509065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:15.509301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:15.509580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:15.598697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] MFA Rules not processed for user `7bd83a97735e4cd1acaebf21f050c334`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:15.600065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:15.683750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:15.729136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 875/1753] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:15.754094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:15.755890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:15.905557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32a7b7-d901-4194-8e40-d458d970405a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:15.971991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:15.972708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:15.972992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:15.973561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:15.990998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:15.992446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:16.007321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 879/1754] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:16.045321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:16.047750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:16.204175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3dea0-82f4-4e67-8f3c-d2e5064396c2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:16.211340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:16.212066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:16.212354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:16.212727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:16.352982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:16.354848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f44e6f-1a9f-413b-8a82-22e01185a502 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:16.381334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:16.393452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5cdbb-e180-4f9a-bfc0-d69355870f60 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:16.404937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 876/1755] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:49:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:49:16.412491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:16.418803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:16.418803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:16.418803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:16.419062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:16.628915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:16.631245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:16.677562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 880/1756] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:17.252793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:18.047177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:18.052934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:18.052934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:18.116802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:18.128674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:18.219650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:18.223570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:18.265074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:18.362844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 877/1757] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jul 16 12:49:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:18.436743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ef9e6-c781-447a-9791-253ed7638325 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:18.443810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:18.445022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:18.445494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:18.445918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] PATH_INFO: `/v3/projects/c25e0ef62eb74f728551f722f3cc548b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:18.447591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25e0ef62eb74f728551f722f3cc548b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:18.457248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:18.612462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:18.614904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:18.643247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e2de-2204-4dae-b62f-28d898cd8bb2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:18.654311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:18.662463 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:18.663010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:18.667839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:18.699742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:18.703851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6136c-1244-4f9f-97f7-eeb5acfbec1d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:18.716387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25e0ef62eb74f728551f722f3cc548b', 'request_id': 'req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:18.744938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 878/1758] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:18.756760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25e0ef62eb74f728551f722f3cc548b', 'request_id': 'req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:18.841869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25e0ef62eb74f728551f722f3cc548b', 'request_id': 'req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:18.909648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] The token cache is being invalidate because project c25e0ef62eb74f728551f722f3cc548b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:18.911052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:18.917704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-000db90e-bb79-4ee4-a09d-ad977ed4ab7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:18.925004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 881/1759] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:18 2022] DELETE /identity/v3/projects/c25e0ef62eb74f728551f722f3cc548b => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:18.930985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:18.931489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:18.932325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:18.995094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:18.998236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:19.217013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:19.217447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:19.217620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 879/1760] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:19.218145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:19.284681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:19.292925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:19.327247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 882/1761] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:19.331561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:19.350556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25661317-f320-464d-86c6-0257f6fb50ad None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:19.370345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:19.375213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:19.375213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:19.375213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:19.375213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:19.375213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:19.387454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:19.444704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 880/1762] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:19.444797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:19.471531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd56af0-7f93-48d9-8f2c-6575daa0385f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:19.477999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:19.479032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:19.479914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:19.480349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:19.481542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:19.484940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:19.647165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:19.656702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 883/1763] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:19 2022] POST /identity/v3/projects => generated 357 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:19.666562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86b68-4f87-447d-abed-f97550b32714 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:19.732047 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:19.802056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:19.802780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:19.807882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:19.809240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:19.813068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:19.815691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:19.951752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 881/1764] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:19 2022] POST /identity/v3/users => generated 279 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:19.956927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:19.965840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:19.981276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282166-0553-4914-9990-44276ea74c6b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:20.007585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:20.009521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:20.011546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:20.011859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:20.014790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:20.024841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:20.066376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 884/1765] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:20.070751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:20.085072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062090b8-f82c-4a4a-b6d0-5a553f86003a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:20.097568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:20.098884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:20.102806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:20.109416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:20.116238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:20.118037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7c7fd-4cb3-49a9-969b-01e6e645aab9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:20.158621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:20.167583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 882/1766] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:20.174315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a5d52-6d09-43c6-aa24-c3e40ba8b57a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:20.205358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:20.206514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:20.209446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:20.209889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] PATH_INFO: `/v3/projects/57024b52ab134df0a46f9bc401053e9c/users/701689ea89f34c3d8ce328215428ea80/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:20.295282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=57024b52ab134df0a46f9bc401053e9c, user_id=701689ea89f34c3d8ce328215428ea80, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:20.309234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:20.443524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 885/1767] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:20 2022] PUT /identity/v3/projects/57024b52ab134df0a46f9bc401053e9c/users/701689ea89f34c3d8ce328215428ea80/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:20.456699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:20.457621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:20.458126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:20.542327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] MFA Rules not processed for user `701689ea89f34c3d8ce328215428ea80`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:20.553064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:20.762619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:20.764208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 883/1768] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:21.000970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02c2b-eb7d-4488-ac8a-31fe86cf88bb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:21.020377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:21.022977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:21.023587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:21.025135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:21.029009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:21.032515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:21.159768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:21.161980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 886/1769] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:20 2022] POST /identity/v3/projects => generated 357 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:21.192439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec299d9-5408-4e30-bb19-0ea5f653f893 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:21.333178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:21.334025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:21.335448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:21.335730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:21.338180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:21.340815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:21.480741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 884/1770] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:21 2022] POST /identity/v3/users => generated 285 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:21.485923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:21.501396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689fe0-c72b-4842-b0f0-02dda866906f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:21.506337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:21.507418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:21.507844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:21.508401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:21.509322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:21.511303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:21.545508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 887/1771] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:21.549861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:21.561822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df20d6-1fe5-4755-8f3b-3275879fd1d2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:21.577771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:21.580055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:21.580512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:21.580938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] PATH_INFO: `/v3/projects/c2bd65816b764637bde9299e07c22a8e/users/f4da56860fdc4830b811e29b4d55b65a/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:21.652611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bd65816b764637bde9299e07c22a8e, user_id=f4da56860fdc4830b811e29b4d55b65a, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:21.664686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:21.813971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 885/1772] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:21 2022] PUT /identity/v3/projects/c2bd65816b764637bde9299e07c22a8e/users/f4da56860fdc4830b811e29b4d55b65a/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:21.818147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:21.832452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93939ba1-1ba7-40d6-9a26-d7ded87559c1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:22.089938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:22.091090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:22.091833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:22.092252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:22.093400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:22.095860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90ab6-f36b-48db-a26a-7a424d6afd20 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:22.136803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 888/1773] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:22.136964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:22.161354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82099b8c-6845-4c8b-b3c5-907227aa3928 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:22.198111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:22.204229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:22.204229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:22.204229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] PATH_INFO: `/v3/projects/c2bd65816b764637bde9299e07c22a8e/users/f4da56860fdc4830b811e29b4d55b65a/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:22.216147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bd65816b764637bde9299e07c22a8e, user_id=f4da56860fdc4830b811e29b4d55b65a, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:22.223921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:22.240745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:22.273997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 886/1774] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:22 2022] PUT /identity/v3/projects/c2bd65816b764637bde9299e07c22a8e/users/f4da56860fdc4830b811e29b4d55b65a/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:22.288319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:22.288602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:22.288879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:22.463201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] MFA Rules not processed for user `f4da56860fdc4830b811e29b4d55b65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:22.474600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:22.655244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:22.814378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 889/1775] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:22.854381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:22.856334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:23.024781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f8d7-8887-43fd-8a74-f8051c4c64c6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:23.042545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:23.044749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:23.046934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:23.047286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:23.086383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:23.086383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:23.121876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 887/1776] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:22 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:23.653782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:23.683787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9821c-6fc7-4975-ad93-f7059983e588 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:23.696255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:23.699551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:23.699551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:23.699551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:23.699551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:23.843661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:23.845916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d625f1-bfb4-4c36-a85f-15bce821e28b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:23.862558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 890/1777] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:23.946839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:23.949282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:23.970339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d69a-aec2-4a34-9aa2-dc68f665f23d service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:23.975187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:23.979052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:23.979052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:23.979052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:24.143457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:24.146536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571b3ee-d4f5-4b94-8c60-24be93fa84d7 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:24.175158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 888/1778] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:24.909541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:24.910091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:24.910091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:24.911730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 891/1779] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:24.927223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356328a-dd97-4b61-acee-51cf732863f2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:24.927428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356328a-dd97-4b61-acee-51cf732863f2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:24.931383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356328a-dd97-4b61-acee-51cf732863f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:24.959445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b356328a-dd97-4b61-acee-51cf732863f2 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:24.963898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b356328a-dd97-4b61-acee-51cf732863f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:25.153160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 889/1780] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:25.437912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:25.440464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:25.483000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e9790-38a8-4dbf-972c-7f051ce09f24 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:25.492755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:25.493996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:25.494500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:25.495012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:25.530021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:25.531866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaad2ea-4439-403a-8128-49181769f098 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:25.532163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaad2ea-4439-403a-8128-49181769f098 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:25.532437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaad2ea-4439-403a-8128-49181769f098 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:25.534414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:25.537601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 890/1781] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:25.566223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 892/1782] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:25.569955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:25.570267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:25.570548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:25.610149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:25.611385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:25.647466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 891/1783] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:25.700052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:25.715195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46e069-3784-4f77-9912-fd124caad5e6 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:25.722811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:25.724016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:25.724478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:25.724931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:25.737025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:25.739941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a152455-31d4-49bf-ac8d-d0f6a16035b4 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:25.754892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 893/1784] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:31.772655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:31.776583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:31.824219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:31.879880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5ba76-2304-4728-b77f-c855abdded89 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:31.885193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:31.888323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:31.888323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:31.888323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] PATH_INFO: `/v3/users/f25a8a9d2d5a453f820aad105c31dd1a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:31.888323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25a8a9d2d5a453f820aad105c31dd1a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:31.891515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:32.111241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25a8a9d2d5a453f820aad105c31dd1a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:32.119255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25a8a9d2d5a453f820aad105c31dd1a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:32.142406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25a8a9d2d5a453f820aad105c31dd1a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:32.142406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25a8a9d2d5a453f820aad105c31dd1a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:32.255931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25a8a9d2d5a453f820aad105c31dd1a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:32.274079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 892/1785] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:31 2022] DELETE /identity/v3/users/f25a8a9d2d5a453f820aad105c31dd1a => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:32.279103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:32.279623 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:32.280212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:32.326607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:32.330689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:32.469408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 894/1786] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:32.526569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:32.528821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:32.650773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f41e5-2aae-4344-bd0d-92e417bdd3bc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:32.658069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:32.659080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:32.660105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:32.660682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:32.675028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:32.677659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3159e-338f-409d-87c0-1d12ce8ecd8f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:32.692043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 893/1787] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:34.388973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:34.388973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:34.404886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22359-9cfe-4826-b90a-1fef7cc36d13 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:34.411539 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:34.414626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:34.414626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:34.414626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] PATH_INFO: `/v3/users/7f16a5fd7b814a5aa700c31e82a7c705` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:34.414985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f16a5fd7b814a5aa700c31e82a7c705)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:34.417251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:34.706324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:34.731878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f16a5fd7b814a5aa700c31e82a7c705'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:34.746492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f16a5fd7b814a5aa700c31e82a7c705'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:34.748130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f16a5fd7b814a5aa700c31e82a7c705'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:34.749744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f16a5fd7b814a5aa700c31e82a7c705'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:34.758119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f16a5fd7b814a5aa700c31e82a7c705'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:34.871000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 895/1788] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:34 2022] DELETE /identity/v3/users/7f16a5fd7b814a5aa700c31e82a7c705 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:34.876795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:34.877370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:34.877370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:34.882306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:34.883771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:34.900927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:34.902777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:34.956130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:34.977291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3e1fc-8390-4978-9d82-eef0f9ed49c0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:34.980331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 894/1789] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:34.986277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:34.987258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:34.987615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:34.987965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] PATH_INFO: `/v3/projects/64812b0063ce47fc9dfa84a094b448e4` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:34.989355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64812b0063ce47fc9dfa84a094b448e4)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:34.991463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:34.991681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:34.993136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:35.054260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76005a-ee02-4ac3-9ae9-4878d3657431 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:35.058247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:35.062772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:35.062772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:35.062772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:35.069724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:35.071153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:35.080783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 895/1790] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:35.125374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64812b0063ce47fc9dfa84a094b448e4', 'request_id': 'req-605383b8-f325-422f-b541-5f4d304a26f6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.131670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64812b0063ce47fc9dfa84a094b448e4', 'request_id': 'req-605383b8-f325-422f-b541-5f4d304a26f6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.208430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64812b0063ce47fc9dfa84a094b448e4', 'request_id': 'req-605383b8-f325-422f-b541-5f4d304a26f6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.220024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] The token cache is being invalidate because project 64812b0063ce47fc9dfa84a094b448e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:35.220318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.220976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.222583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 896/1791] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:34 2022] DELETE /identity/v3/projects/64812b0063ce47fc9dfa84a094b448e4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:35.226669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:35.236929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc578f7a-8128-4699-aa06-701bafb836b5 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:35.294692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:35.296310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:35.296588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:35.296861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] PATH_INFO: `/v3/users/515dccd9e4544808b754d24ee2da9472` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:35.298104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515dccd9e4544808b754d24ee2da9472)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:35.300110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:35.457660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515dccd9e4544808b754d24ee2da9472'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.465601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515dccd9e4544808b754d24ee2da9472'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.479778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515dccd9e4544808b754d24ee2da9472'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.480523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515dccd9e4544808b754d24ee2da9472'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.546595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515dccd9e4544808b754d24ee2da9472'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.553640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 896/1792] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:35 2022] DELETE /identity/v3/users/515dccd9e4544808b754d24ee2da9472 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:35.638441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:35.640932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:35.660184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605383b8-f325-422f-b541-5f4d304a26f6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:35.705430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:35.722151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:35.723100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:35.723399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:35.723676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] PATH_INFO: `/v3/projects/99aeabea578d48a6b8bd6107154b0a8d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:35.724878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99aeabea578d48a6b8bd6107154b0a8d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:35.726880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:35.865762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99aeabea578d48a6b8bd6107154b0a8d', 'request_id': 'req-07c3514b-0eb5-4b3f-893d-fd1334447379'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.876521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99aeabea578d48a6b8bd6107154b0a8d', 'request_id': 'req-07c3514b-0eb5-4b3f-893d-fd1334447379'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.962515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99aeabea578d48a6b8bd6107154b0a8d', 'request_id': 'req-07c3514b-0eb5-4b3f-893d-fd1334447379'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.978379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] The token cache is being invalidate because project 99aeabea578d48a6b8bd6107154b0a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:35.978379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.979584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:35.981604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 897/1793] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:35 2022] DELETE /identity/v3/projects/99aeabea578d48a6b8bd6107154b0a8d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:35.986292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:35.987799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:36.004078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c0b2c-6726-43bb-940d-0a0c4c018127 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:36.007885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:36.078666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:36.083478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:36.083478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:36.083478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] PATH_INFO: `/v3/users/7bd83a97735e4cd1acaebf21f050c334` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:36.083478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd83a97735e4cd1acaebf21f050c334)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:36.084204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:36.130441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:36.131960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:36.145605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3514b-0eb5-4b3f-893d-fd1334447379 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:36.151386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:36.152263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:36.152546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:36.152850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] PATH_INFO: `/v3/projects/dc1516186d404516921dc4c047a14e10` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:36.154147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1516186d404516921dc4c047a14e10)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:36.157345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:36.314250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd83a97735e4cd1acaebf21f050c334'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.322876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd83a97735e4cd1acaebf21f050c334'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.332001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1516186d404516921dc4c047a14e10', 'request_id': 'req-9c049e93-ae0c-4e0f-b466-ac3c924574fd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.339667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd83a97735e4cd1acaebf21f050c334'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.340471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd83a97735e4cd1acaebf21f050c334'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.340752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1516186d404516921dc4c047a14e10', 'request_id': 'req-9c049e93-ae0c-4e0f-b466-ac3c924574fd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.430921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1516186d404516921dc4c047a14e10', 'request_id': 'req-9c049e93-ae0c-4e0f-b466-ac3c924574fd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.443746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] The token cache is being invalidate because project dc1516186d404516921dc4c047a14e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:36.444035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.444703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.446263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 898/1794] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:36 2022] DELETE /identity/v3/projects/dc1516186d404516921dc4c047a14e10 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:36.449935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:36.451325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:36.487934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd83a97735e4cd1acaebf21f050c334'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.494673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 897/1795] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:35 2022] DELETE /identity/v3/users/7bd83a97735e4cd1acaebf21f050c334 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:36.506971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:36.519172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c049e93-ae0c-4e0f-b466-ac3c924574fd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:36.580937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:36.582253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:36.582454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:36.582681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] PATH_INFO: `/v3/users/63a1a6c295974f43bb9ae8ce3ae220df` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:36.584050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a1a6c295974f43bb9ae8ce3ae220df)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:36.586415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:36.772607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a1a6c295974f43bb9ae8ce3ae220df'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.794370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a1a6c295974f43bb9ae8ce3ae220df'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.796432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a1a6c295974f43bb9ae8ce3ae220df'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.797206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a1a6c295974f43bb9ae8ce3ae220df'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.805233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-38d15cc9-edd9-4fa7-809e-979afeba3748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a1a6c295974f43bb9ae8ce3ae220df'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:36.877709 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 899/1796] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:36 2022] DELETE /identity/v3/users/63a1a6c295974f43bb9ae8ce3ae220df => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:36.942118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:36.942118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:36.951557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca4c2c-8109-4cc7-a18e-ea2913996aec admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.014734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.015738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.015853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.016095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] PATH_INFO: `/v3/projects/7e890c573f7d446d864314719b1b69a4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.017332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e890c573f7d446d864314719b1b69a4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.019218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.109257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:37.133065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e890c573f7d446d864314719b1b69a4', 'request_id': 'req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.141238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e890c573f7d446d864314719b1b69a4', 'request_id': 'req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.214071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e890c573f7d446d864314719b1b69a4', 'request_id': 'req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.236750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] The token cache is being invalidate because project 7e890c573f7d446d864314719b1b69a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:37.237061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.237766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4a442a5f-ed4b-4366-87e2-36cd9cc8f593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.239381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 898/1797] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:36 2022] DELETE /identity/v3/projects/7e890c573f7d446d864314719b1b69a4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:37.253230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.253601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.253956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.278125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:37.279540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.361610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 900/1798] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:37.369469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.369929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.370730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.393685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:37.395017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.407086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 899/1799] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:37.412977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.422873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89897cd-4865-4cfd-a55e-5379738316c5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.427565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.431267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.431267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.431267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.431267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:37.431267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.431671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.444127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 901/1800] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:37.454335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.466841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300e497-b80a-41f5-81d3-2ae2320a0b2b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.469857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.486650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.487153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.487153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.487153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.487153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.487414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.487414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.487535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a23e-dc7b-4ffd-89de-3f66f246cf5b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.492396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.493304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.493600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.494740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] PATH_INFO: `/v3/projects/a1ccd2da06a543fe813b99c70840338a` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.496450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ccd2da06a543fe813b99c70840338a)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.501599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.505421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:37.571471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 900/1801] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:37.575553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.643679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ccd2da06a543fe813b99c70840338a', 'request_id': 'req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.652458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191c483-ebed-429c-82c1-1ea0541e247d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.661898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ccd2da06a543fe813b99c70840338a', 'request_id': 'req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.717751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.718320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.718603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.718876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.724876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.724876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.739574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ccd2da06a543fe813b99c70840338a', 'request_id': 'req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.752027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] The token cache is being invalidate because project a1ccd2da06a543fe813b99c70840338a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:37.752317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.752971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d9f57f3c-1ede-40f0-bd80-81fc7c0b231b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:37.754596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 902/1802] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:37 2022] DELETE /identity/v3/projects/a1ccd2da06a543fe813b99c70840338a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:37.760379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aecc6-b2bb-4baa-9643-fcd10c69afa4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.760672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aecc6-b2bb-4baa-9643-fcd10c69afa4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.760932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aecc6-b2bb-4baa-9643-fcd10c69afa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.778769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-be0aecc6-b2bb-4baa-9643-fcd10c69afa4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:37.780583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be0aecc6-b2bb-4baa-9643-fcd10c69afa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.788360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 903/1803] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:37.796894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.797264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.797483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.809927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 901/1804] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:37.814504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.816261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.836838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96314c1-9659-4b17-85b8-198d1f0b65fe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.844837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:37.846063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:37.849506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.850185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.850465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.850740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.851466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.853096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.875472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 904/1805] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:37.875661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 902/1806] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:37.876561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.882688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.895209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851a359-64ec-42b3-90ce-a5637bf45352 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.896504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8a8c-c82f-4839-a2e2-f9ada3742090 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.901203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.905649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.906221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.906221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.906387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.906466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.906545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.906545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.906684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.906759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:37.906759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.906879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.906946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.915829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:37.917274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 905/1807] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:37.920499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 903/1808] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:37.922215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.925093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:37.933654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b820cb7-8593-4709-9b37-197c46173423 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.938387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e078b-a858-49e5-adc2-2e3474f1786d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:37.940027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.940930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.941722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.941722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] PATH_INFO: `/v3/projects/b8346a803a14472b96c75b20091141fd/users/fe3d875e05774d499bd4f03ff31cc7e9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.946026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:37.946694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:37.946980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:37.947257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:37.947958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.952506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b99598-a957-46f0-bc6f-4973ab248b4b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:37.958339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8346a803a14472b96c75b20091141fd, user_id=fe3d875e05774d499bd4f03ff31cc7e9, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:37.961481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.061305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 906/1809] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:37 2022] PUT /identity/v3/projects/b8346a803a14472b96c75b20091141fd/users/fe3d875e05774d499bd4f03ff31cc7e9/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:38.063419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 904/1810] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:37 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:38.067330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.073088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.073442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.074729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.101600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] MFA Rules not processed for user `fe3d875e05774d499bd4f03ff31cc7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:38.102791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.139951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63991fd-612e-459f-ab4a-0a2769f40ad4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.201999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:38.206620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 905/1811] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:38.215253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.215946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.216224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.216493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.216993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.217366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.218464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.219211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.281659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd951f-7ce0-4b4e-916f-fa74fb9f6461 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.285523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.286170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.286475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.286721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.301136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.302606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.313742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 906/1812] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:38.323467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 907/1813] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:38.328043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.344549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337c2fd-510c-40ed-b8d2-ee378d73c51f service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.350531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.350633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.350926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.351356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.351840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.352586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.355312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.357178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4e82d-75ba-4c98-9b79-0a5c8794acf0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.363849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.364916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.365346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.365907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] PATH_INFO: `/v3/users/fe3d875e05774d499bd4f03ff31cc7e9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.367740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3d875e05774d499bd4f03ff31cc7e9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.369696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 907/1814] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:38.373231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.373615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.385843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d695-fd58-4106-adfa-47535feb197d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.393237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.393931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.394417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.394771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.397157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.398935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.428869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 908/1815] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:38.434239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.468947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb05a1-053c-4c29-8429-b5f574d93b1c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.474405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.475818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.476160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.476506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] PATH_INFO: `/v3/projects/c6f2d79c3ec54c34ad5c58b42e27e659/users/0efdb5ae660d4c059b326a3ebb65c305/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.513448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f2d79c3ec54c34ad5c58b42e27e659, user_id=0efdb5ae660d4c059b326a3ebb65c305, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.517831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff80b73-144b-4b3f-bb76-44322e36a974 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.569299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3d875e05774d499bd4f03ff31cc7e9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:38.583318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3d875e05774d499bd4f03ff31cc7e9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:38.586241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3d875e05774d499bd4f03ff31cc7e9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:38.586241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3d875e05774d499bd4f03ff31cc7e9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:38.595765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1966ad94-0071-4ca5-89b8-835bc6f14809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3d875e05774d499bd4f03ff31cc7e9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:38.624966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 909/1816] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:38 2022] PUT /identity/v3/projects/c6f2d79c3ec54c34ad5c58b42e27e659/users/0efdb5ae660d4c059b326a3ebb65c305/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:38.636447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.636811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.637254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.667084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] MFA Rules not processed for user `0efdb5ae660d4c059b326a3ebb65c305`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:38.668445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.706117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 908/1817] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:38 2022] DELETE /identity/v3/users/fe3d875e05774d499bd4f03ff31cc7e9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:38.710575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.710828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.711128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.729267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:38.730474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.744884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 910/1818] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:38.751027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.752780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.800113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 909/1819] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:38.810408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.812656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:38.820949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983fe1c-14f5-4981-ab71-897c1dfd55c0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.826670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.827375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.827660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.827939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.828635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.830574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.865677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:38.875921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339357ec-a7ae-437e-85c3-3938d7cf24b1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:38.879662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:38.880323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:38.880610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:38.880972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:38.890306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:38.891725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:38.903410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 910/1820] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:38.932891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 911/1821] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:38.934123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:38.944538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcad5f1-e9c7-4468-b1c7-201bbcd3b731 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.002021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.002714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.002983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.003257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.003944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.006006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.068534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 911/1822] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:38 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:39.070001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.081099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9982dc-73d6-4889-95e0-9425dbadf84e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.085820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.086490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.086775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.087051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.087776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.089462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.103777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 912/1823] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:39.105277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.116606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e20c5-2ddb-4254-99f1-8bf5b4b8cc97 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.123987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.123987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.123987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.123987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.124313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.125804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0d03d-637a-49d0-bad7-d8b7049dc7a0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.138773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:39.145974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 912/1824] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:39.148783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.163905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88d2c-7f05-4a32-8167-7bfeaa78f655 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.165368 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.166277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.166563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.167171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] PATH_INFO: `/v3/projects/6bef57cb40c14a10b9bfe804cfe44e20/users/cc538619190542c3b2656df0d5bd2417/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.205097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bef57cb40c14a10b9bfe804cfe44e20, user_id=cc538619190542c3b2656df0d5bd2417, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.205097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.271701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 913/1825] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:39 2022] PUT /identity/v3/projects/6bef57cb40c14a10b9bfe804cfe44e20/users/cc538619190542c3b2656df0d5bd2417/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:39.283547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.283843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.284131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.311113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] MFA Rules not processed for user `cc538619190542c3b2656df0d5bd2417`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:39.312336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:39.376516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 913/1826] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:39.381212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.404968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.406642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:39.462874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125ec1b-9421-498b-8464-2dfda3d1c8f8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.469838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.470652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.471145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.471340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.472038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.474077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.483035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608467-9844-4336-bb87-6e8d57ffb41f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.487223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:39.489179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.490047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.490331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.490605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] PATH_INFO: `/v3/projects/b8346a803a14472b96c75b20091141fd` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.491800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8346a803a14472b96c75b20091141fd)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.493975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.497694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:39.608148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 914/1827] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:39 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:39.612228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.648898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8346a803a14472b96c75b20091141fd', 'request_id': 'req-754b7d4a-2758-4786-bb26-8187961db0a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:39.657714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8346a803a14472b96c75b20091141fd', 'request_id': 'req-754b7d4a-2758-4786-bb26-8187961db0a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:39.732890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcd5c9-90b0-4a75-bc7e-f9e54d37c4a9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.779960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8346a803a14472b96c75b20091141fd', 'request_id': 'req-754b7d4a-2758-4786-bb26-8187961db0a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:39.807863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] The token cache is being invalidate because project b8346a803a14472b96c75b20091141fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:39.809669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:39.811567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-754b7d4a-2758-4786-bb26-8187961db0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:39.813520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 914/1828] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:39 2022] DELETE /identity/v3/projects/b8346a803a14472b96c75b20091141fd => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:39.818878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9f25d-7d15-41e2-9c1d-cd5925d36653 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.819175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9f25d-7d15-41e2-9c1d-cd5925d36653 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.819454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9f25d-7d15-41e2-9c1d-cd5925d36653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.834374 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.835348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.835876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.836317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.837207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.840191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.861066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e3e9f25d-7d15-41e2-9c1d-cd5925d36653 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:39.863386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e3e9f25d-7d15-41e2-9c1d-cd5925d36653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:39.883281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 915/1829] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:39.891870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.892337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.892618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.927867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:39.930139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:39.948199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 916/1830] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:39.952958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.959252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 915/1831] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:39 2022] POST /identity/v3/users => generated 296 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:39.963704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:39.965583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:39.971818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c7c74-c65d-4005-9a4a-78592ef7c052 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:39.989874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:39.989874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:39.989874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:39.989874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:39.990168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:39.990168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:39.992326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:39.993952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b8a95-6f3d-4328-b9be-de0f73b17339 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.001333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.002010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.002297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.002624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.009808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.015691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.019968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 917/1832] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:49:40.024948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.034853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 916/1833] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:40.035899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.043916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b16a57-97b7-4677-9160-c25ee4f03113 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.051766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.052449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55662e8-5cb6-48ff-8fd8-d2890a6b848c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.053012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.053517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.053984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.055278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.058382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d0ede-cbaa-46d3-a7f0-1c05c91d4975 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.059172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.060218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.060796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.061187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] PATH_INFO: `/v3/projects/e13970ee2e6b41fbaf83c430dcc87575/users/bcda4ce7dc0749719407ea44b34cdb31/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.079917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13970ee2e6b41fbaf83c430dcc87575, user_id=bcda4ce7dc0749719407ea44b34cdb31, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.084410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.183260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 918/1834] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:40 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:40.184615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e583-2810-46fb-97c7-f2e2e8014b5b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.186812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e583-2810-46fb-97c7-f2e2e8014b5b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.186812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e583-2810-46fb-97c7-f2e2e8014b5b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.187515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 919/1835] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:49:40.189931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e583-2810-46fb-97c7-f2e2e8014b5b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.207418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 917/1836] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:40 2022] PUT /identity/v3/projects/e13970ee2e6b41fbaf83c430dcc87575/users/bcda4ce7dc0749719407ea44b34cdb31/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:40.212800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.309601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:40.318578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e583-2810-46fb-97c7-f2e2e8014b5b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.323483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.325570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.325570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.325853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c59c2e-9159-43fc-be70-b8b464c8b831 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.326353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.326353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.328126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.330821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.331777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.332167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.332515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.333434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.336659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ec80f-a6d0-4531-86b0-3ad36187de23 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.349088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 918/1837] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:40.349922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.350219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.350502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.370355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:40.371569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.430175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 920/1838] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:40 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:40.430175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.452257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8035f-a758-4a2d-8460-2df44758cc72 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.461184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.462421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.463037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.463349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] PATH_INFO: `/v3/projects/e13970ee2e6b41fbaf83c430dcc87575/users/bcda4ce7dc0749719407ea44b34cdb31/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.466014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13970ee2e6b41fbaf83c430dcc87575, user_id=bcda4ce7dc0749719407ea44b34cdb31, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.468692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.474402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 919/1839] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:40.480218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 921/1840] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:40 2022] PUT /identity/v3/projects/e13970ee2e6b41fbaf83c430dcc87575/users/bcda4ce7dc0749719407ea44b34cdb31/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:40.489241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.490699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.492679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.510064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619fcd6-cd84-462a-99e0-ac63b8bcbc75 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.524571 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:40.583162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.583976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.584327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.584668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.585519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.587174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503124d8-8146-4ae6-8791-fac415fcdea5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.591046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d821286-4283-403e-a2b9-bf4db05863ce admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.597214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.597871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.598157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.598432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.599252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 920/1841] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:40.600691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.601166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.601378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.611283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.612975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.623975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 922/1842] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:40.624705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.628632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] MFA Rules not processed for user `bcda4ce7dc0749719407ea44b34cdb31`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:40.629831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.634742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626020cf-87c4-4d41-96ca-3c72f10fdf93 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.640421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.641139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.641494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.641843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.642662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.644322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.658965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 923/1843] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:40.660326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.672459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36ea8-0c88-4e13-90a4-0a9aef607945 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.680320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.681725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.682016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.682328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] PATH_INFO: `/v3/projects/d61b15bdc1bd42449ae37abc750e2e9d/users/fb8798dc5940426497a44fbfd85a410b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.699648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d61b15bdc1bd42449ae37abc750e2e9d, user_id=fb8798dc5940426497a44fbfd85a410b, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.703124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e617d3-7250-479d-887d-77c15e7e8e1b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.707519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:40.712085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 921/1844] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:40.724580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.726021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.792644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c590de-8140-4597-bc59-8aa52049ff7c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:40.804112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 924/1845] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:40 2022] PUT /identity/v3/projects/d61b15bdc1bd42449ae37abc750e2e9d/users/fb8798dc5940426497a44fbfd85a410b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:40.806927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.807266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.807553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.843594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] MFA Rules not processed for user `fb8798dc5940426497a44fbfd85a410b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:40.845137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.868348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:40.869000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:40.869308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:40.869584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:40.923216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 925/1846] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:40.934599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:40.936127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:40.943545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:40.944986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:40.956982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 922/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:41.004744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:41.018640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9428e1-33d4-4229-a9d2-85f5f927640b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:41.022266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:41.022944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:41.023221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:41.023496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:41.032868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:41.034442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:41.043593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 926/1848] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:41.068675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:41.070135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:41.082293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f52e-5881-4495-bec5-118415c474aa service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:41.086215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:41.086942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:41.087298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:41.087644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:41.089625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:41.151857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:41.153334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4825b77-3146-49e8-8727-157499dc6ce9 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:41.163483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 923/1849] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:46.492157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:46.555919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec5ad-3ff2-40c8-9ab1-8fd3a1b029cf service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:46.560417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:46.561988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:46.562277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:46.562549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] PATH_INFO: `/v3/users/0efdb5ae660d4c059b326a3ebb65c305` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:46.563736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efdb5ae660d4c059b326a3ebb65c305)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:46.565755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:46.728243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efdb5ae660d4c059b326a3ebb65c305'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:46.741424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efdb5ae660d4c059b326a3ebb65c305'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:46.742146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efdb5ae660d4c059b326a3ebb65c305'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:46.742768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efdb5ae660d4c059b326a3ebb65c305'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:46.750220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efdb5ae660d4c059b326a3ebb65c305'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:46.825595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 927/1850] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:46 2022] DELETE /identity/v3/users/0efdb5ae660d4c059b326a3ebb65c305 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:46.830706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:46.831275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:46.831275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:46.838131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:46.849202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:46.853841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:46.914456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 924/1851] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:46.924734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:46.926346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:46.994996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbe7e5-4950-4ecc-b37c-9f9c2ab3bceb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:46.998331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:46.999546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:46.999826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:47.000282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:47.009913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:47.011352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:47.022262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 928/1852] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:47.602637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:47.604241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:47.619934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b8f63-4f0f-4808-a20d-7bd7cfe2638d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:47.633072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:47.634068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:47.634416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:47.634792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] PATH_INFO: `/v3/projects/c6f2d79c3ec54c34ad5c58b42e27e659` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:47.636123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f2d79c3ec54c34ad5c58b42e27e659)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:47.638187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:47.744028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f2d79c3ec54c34ad5c58b42e27e659', 'request_id': 'req-c00df5ca-f753-4388-b3de-431d0cb7075b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:47.750578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f2d79c3ec54c34ad5c58b42e27e659', 'request_id': 'req-c00df5ca-f753-4388-b3de-431d0cb7075b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:47.830018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f2d79c3ec54c34ad5c58b42e27e659', 'request_id': 'req-c00df5ca-f753-4388-b3de-431d0cb7075b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:47.842983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] The token cache is being invalidate because project c6f2d79c3ec54c34ad5c58b42e27e659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:47.843649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:47.844296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:47.846184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 925/1853] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:47 2022] DELETE /identity/v3/projects/c6f2d79c3ec54c34ad5c58b42e27e659 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:47.853260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:47.856901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:47.862119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00cb57-4c70-4bfa-b1a0-1b25b5f283ec service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:47.942072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:47.943471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:47.944002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:47.944472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] PATH_INFO: `/v3/users/cc538619190542c3b2656df0d5bd2417` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:47.946291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc538619190542c3b2656df0d5bd2417)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:47.948583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:48.091273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:48.093307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:48.105988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df5ca-f753-4388-b3de-431d0cb7075b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:48.112419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:48.113567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:48.113747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc538619190542c3b2656df0d5bd2417'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.113862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:48.114121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] PATH_INFO: `/v3/users/701689ea89f34c3d8ce328215428ea80` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:48.115348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=701689ea89f34c3d8ce328215428ea80)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:48.117448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:48.139992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc538619190542c3b2656df0d5bd2417'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.140930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc538619190542c3b2656df0d5bd2417'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.141789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc538619190542c3b2656df0d5bd2417'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.151103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-693bb09d-a584-4cdb-9c7d-dc0f7ceac662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc538619190542c3b2656df0d5bd2417'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.295055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 929/1854] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:47 2022] DELETE /identity/v3/users/cc538619190542c3b2656df0d5bd2417 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:48.334906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701689ea89f34c3d8ce328215428ea80'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.348360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701689ea89f34c3d8ce328215428ea80'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.365701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701689ea89f34c3d8ce328215428ea80'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.366736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701689ea89f34c3d8ce328215428ea80'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.494437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701689ea89f34c3d8ce328215428ea80'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:48.497226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:48.501991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 926/1855] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:48 2022] DELETE /identity/v3/users/701689ea89f34c3d8ce328215428ea80 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:48.506641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:48.506934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:48.507213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:48.524536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:48.525816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:48.609199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 930/1856] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:48.620469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:48.622056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:48.716141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517712d-f23a-4a89-8988-82a7dd950803 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:48.725675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:48.726512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:48.726865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:48.727255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:48.739078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:48.740899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:48.760187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 927/1857] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:49.049266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:49.051501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:49.072695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f93d8a-5e84-4857-8cf9-9fec1982eb33 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:49.079410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:49.080471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:49.080877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:49.081324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] PATH_INFO: `/v3/projects/6bef57cb40c14a10b9bfe804cfe44e20` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:49.082899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bef57cb40c14a10b9bfe804cfe44e20)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:49.084867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:49.107640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:49.227761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bef57cb40c14a10b9bfe804cfe44e20', 'request_id': 'req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.234894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bef57cb40c14a10b9bfe804cfe44e20', 'request_id': 'req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.316672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bef57cb40c14a10b9bfe804cfe44e20', 'request_id': 'req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.333071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] The token cache is being invalidate because project 6bef57cb40c14a10b9bfe804cfe44e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:49.333459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.334124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.338779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 931/1858] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:49 2022] DELETE /identity/v3/projects/6bef57cb40c14a10b9bfe804cfe44e20 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:49.346183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:49.355393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:49.368213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:49.371791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a4c3-1be0-442b-bd4f-0ce66d0e2aaf service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:49.465122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:49.466577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:49.467293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:49.467293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] PATH_INFO: `/v3/users/bcda4ce7dc0749719407ea44b34cdb31` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:49.469978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcda4ce7dc0749719407ea44b34cdb31)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:49.473878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:49.484202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1ec4-9e40-429b-92f8-6a4d3074b4a0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:49.501371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:49.501371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:49.501371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:49.501705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] PATH_INFO: `/v3/projects/57024b52ab134df0a46f9bc401053e9c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:49.502817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57024b52ab134df0a46f9bc401053e9c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:49.504896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:49.690162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcda4ce7dc0749719407ea44b34cdb31'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.692527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57024b52ab134df0a46f9bc401053e9c', 'request_id': 'req-fb8377cc-4f4f-4630-a8d8-a949cb01c677'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.699083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcda4ce7dc0749719407ea44b34cdb31'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.700237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57024b52ab134df0a46f9bc401053e9c', 'request_id': 'req-fb8377cc-4f4f-4630-a8d8-a949cb01c677'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.716665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcda4ce7dc0749719407ea44b34cdb31'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.717988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcda4ce7dc0749719407ea44b34cdb31'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.790768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57024b52ab134df0a46f9bc401053e9c', 'request_id': 'req-fb8377cc-4f4f-4630-a8d8-a949cb01c677'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.825245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] The token cache is being invalidate because project 57024b52ab134df0a46f9bc401053e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:49.828563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.828870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.838141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 932/1859] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:49 2022] DELETE /identity/v3/projects/57024b52ab134df0a46f9bc401053e9c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:49.840330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:49.848276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcda4ce7dc0749719407ea44b34cdb31'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:49.849213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:49.855288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 928/1860] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:49 2022] DELETE /identity/v3/users/bcda4ce7dc0749719407ea44b34cdb31 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:49.922893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:49.961826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8377cc-4f4f-4630-a8d8-a949cb01c677 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:50.054492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:50.055832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:50.056209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:50.057027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] PATH_INFO: `/v3/users/f4da56860fdc4830b811e29b4d55b65a` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:50.058042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4da56860fdc4830b811e29b4d55b65a)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:50.060078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:50.266697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4da56860fdc4830b811e29b4d55b65a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:50.290312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4da56860fdc4830b811e29b4d55b65a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:50.291116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4da56860fdc4830b811e29b4d55b65a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:50.291731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4da56860fdc4830b811e29b4d55b65a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:50.308263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4da56860fdc4830b811e29b4d55b65a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:50.381213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 933/1861] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:49 2022] DELETE /identity/v3/users/f4da56860fdc4830b811e29b4d55b65a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:50.844788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:50.846400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:50.864021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd96e-4d7b-4daf-87e5-91c6ff2b93d9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:50.893805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:50.933408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:50.934273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:50.934552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:50.934821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] PATH_INFO: `/v3/projects/e13970ee2e6b41fbaf83c430dcc87575` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:50.936016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13970ee2e6b41fbaf83c430dcc87575)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:50.938122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:50.959740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:50.961247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:50.988981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd7aea-48e6-44ba-bdae-c69daa80d2fe admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:50.994953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:50.996148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:50.996417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:50.996884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] PATH_INFO: `/v3/projects/c2bd65816b764637bde9299e07c22a8e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:50.998829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bd65816b764637bde9299e07c22a8e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.000849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.053650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13970ee2e6b41fbaf83c430dcc87575', 'request_id': 'req-7a32f05a-c685-4d43-982f-4dfb996c5b02'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.060219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13970ee2e6b41fbaf83c430dcc87575', 'request_id': 'req-7a32f05a-c685-4d43-982f-4dfb996c5b02'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.112724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13970ee2e6b41fbaf83c430dcc87575', 'request_id': 'req-7a32f05a-c685-4d43-982f-4dfb996c5b02'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.115435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bd65816b764637bde9299e07c22a8e', 'request_id': 'req-bbe76b98-6722-4ac9-972b-f4fe816f0df1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.122580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bd65816b764637bde9299e07c22a8e', 'request_id': 'req-bbe76b98-6722-4ac9-972b-f4fe816f0df1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.124793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] The token cache is being invalidate because project e13970ee2e6b41fbaf83c430dcc87575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:51.125108 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.125765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a32f05a-c685-4d43-982f-4dfb996c5b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.127396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 929/1862] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:50 2022] DELETE /identity/v3/projects/e13970ee2e6b41fbaf83c430dcc87575 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:51.127813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:51.133609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f25830-f7ae-4eb1-914c-220631d967d2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.133942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f25830-f7ae-4eb1-914c-220631d967d2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.134248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f25830-f7ae-4eb1-914c-220631d967d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.153514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-60f25830-f7ae-4eb1-914c-220631d967d2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:51.154846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-60f25830-f7ae-4eb1-914c-220631d967d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:51.176564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bd65816b764637bde9299e07c22a8e', 'request_id': 'req-bbe76b98-6722-4ac9-972b-f4fe816f0df1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.190077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] The token cache is being invalidate because project c2bd65816b764637bde9299e07c22a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:51.190468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.191422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bbe76b98-6722-4ac9-972b-f4fe816f0df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:51.193395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 934/1863] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:50 2022] DELETE /identity/v3/projects/c2bd65816b764637bde9299e07c22a8e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:51.197781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6841-fabe-4382-8cd9-53569edbdb9a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.198138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6841-fabe-4382-8cd9-53569edbdb9a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.198493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6841-fabe-4382-8cd9-53569edbdb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.218013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b89b6841-fabe-4382-8cd9-53569edbdb9a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:51.220111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b89b6841-fabe-4382-8cd9-53569edbdb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:51.225357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 930/1864] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:51.228315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 935/1865] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:51.235815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.236119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.236428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.241741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.244555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.244913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.260628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:51.261875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:51.267393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:51.270849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:51.273254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 936/1866] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:51.276111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.285544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 931/1867] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:51.286508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea406ff0-5f05-4d73-8707-37828d8f9893 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.300110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.300258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.300258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.300258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.300258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.300592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:51.300592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.300592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.300850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6fb03-f438-4d73-8274-b5b08c327bfb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.305142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.306195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.306566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.306918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.307726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:51.308109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.312361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 937/1868] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:49:51.312533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.313675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.328428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 932/1869] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:49:51.328992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f898302-c645-4721-a85e-f5e5388cb959 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.329370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.338881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.339545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.339677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbf632-9316-452c-8cf0-7b64b92a2234 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.339824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.340091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.340903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.342975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.344645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.345561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.346009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.346435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.347199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.349530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.428752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 938/1870] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:51.434505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.451953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 933/1871] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:51.456077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.474937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:51.533086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1982036-12d4-48b9-b182-3e1446a3bc4d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.539361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f33d70-5602-4243-9f68-8e1d2ee560a2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.541710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.542403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.542711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.543032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.546658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.546809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.547957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.548371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.548875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.549672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.552673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.568202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.654615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 934/1872] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:51.658622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.674962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 939/1873] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:51.676798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.681417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d3f22-c3e3-4463-97cc-4f4868361348 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.689318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098db37-3de7-4cb6-809b-b663954691c5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.690389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.691067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.691357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.691638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.692369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.694018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.695233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.696250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.697145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.697422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.698154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.699807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.705098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 935/1874] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:51.709338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.715014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 940/1875] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:51.719851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.727794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba9d2-4909-4114-bd8a-7c2a652a036a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.731425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f00bf-10b4-4b58-881c-b384c1bd2205 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.738210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.739000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.739286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.739563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.740686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.742950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.755855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 941/1876] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:51.756711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.757743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.758019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.758294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.759025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.760371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.761140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.773137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7ef8a-fa4f-44ee-bff5-cc731e5d8929 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.775208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 936/1877] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:51.778755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.780455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.780725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.781011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] PATH_INFO: `/v3/projects/8952fffef8804b3590800265465bd60e/users/4d55ff81b28443a9a200488a089a4f5e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.807171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8952fffef8804b3590800265465bd60e, user_id=4d55ff81b28443a9a200488a089a4f5e, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.809934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:51.810227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a727-5dfb-4944-80bd-394581695536 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.840149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1175-651a-42df-b6e7-7fec776cf210 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:51.851729 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:51.854944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.854944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.856034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] PATH_INFO: `/v3/projects/f5d55db2d92941c1833c93cd80acd336/users/126305ecdeab43cfabf009d2d93a7bb3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.886440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d55db2d92941c1833c93cd80acd336, user_id=126305ecdeab43cfabf009d2d93a7bb3, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:51.889993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec1b3f-3f08-44c6-927f-b90ce44a14f5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:51.931880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 942/1878] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:51 2022] PUT /identity/v3/projects/8952fffef8804b3590800265465bd60e/users/4d55ff81b28443a9a200488a089a4f5e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:51.943498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:51.943734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:51.944081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:51.994828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] MFA Rules not processed for user `4d55ff81b28443a9a200488a089a4f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:51.997071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:52.010865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 937/1879] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:51 2022] PUT /identity/v3/projects/f5d55db2d92941c1833c93cd80acd336/users/126305ecdeab43cfabf009d2d93a7bb3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:52.026422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.026922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.027215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.087616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] MFA Rules not processed for user `126305ecdeab43cfabf009d2d93a7bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:52.088636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:52.090953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:52.149581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 943/1880] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:52.152737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.231954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 938/1881] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:52.233605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.259928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811b66f-11d7-42a9-8cfd-1196c5f14985 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.300325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:52.327043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc92b2c-0340-4f2c-8d1a-7466777beb8e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.339713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.339713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.339713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.339713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.340469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.340554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.340661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.340746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.340746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.340746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.340746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.341448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.449022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 944/1882] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:52 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:52.449179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 939/1883] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:52 2022] POST /identity/v3/projects => generated 381 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:52.451133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.452499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.529394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968451-078c-4b2b-9d21-ec29a022e999 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.533445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072d0cc-5b19-47fb-8d42-7cbfe5da9f17 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.534483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.535176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.535440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.535708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.536399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.538208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.541548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.541652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.541652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.541652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.541652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.542077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.547585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:52.649154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 945/1884] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:52 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:52.655249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.679569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1a2b-f917-45e5-a239-0b0acd5ed704 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.692090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 940/1885] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:52 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:49:52.693197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.697843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.698851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.699472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.699913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.703371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.705060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.705682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8522-d458-4049-b38e-bc8fde4a42ab admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.714675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.715620 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.715981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.716488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.723392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 946/1886] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:52.723544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.723544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.726815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.735396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 941/1887] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:52.739326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.751875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a6016-a0b9-45c2-abbd-6b8b1131ada7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.758231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.759214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.759802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.760160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] PATH_INFO: `/v3/projects/1b05224f404f4f018926be2d90241768/users/8a8adacdeaeb4a639b4a6204817dcd74/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.771804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930697e-dede-4079-bc83-c28032608ac2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:52.784012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:52.785251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:52.785537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:52.786224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] PATH_INFO: `/v3/projects/63d5a24a95f344fa9589775a5c756759/users/9abc613fc4f842a5a53d840e3e9e9ad1/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:52.801301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b05224f404f4f018926be2d90241768, user_id=8a8adacdeaeb4a639b4a6204817dcd74, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.806193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d5a24a95f344fa9589775a5c756759, user_id=9abc613fc4f842a5a53d840e3e9e9ad1, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:52.807518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.809410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:52.885369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 947/1888] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:52 2022] PUT /identity/v3/projects/63d5a24a95f344fa9589775a5c756759/users/9abc613fc4f842a5a53d840e3e9e9ad1/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:52.893024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:52.911890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 942/1889] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:52 2022] PUT /identity/v3/projects/1b05224f404f4f018926be2d90241768/users/8a8adacdeaeb4a639b4a6204817dcd74/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:52.917413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.017408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5144-e08b-4b59-a9dc-7327d8b256b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.040626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d53b8-d01c-4c44-96cd-58f76b006cea admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.047057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.047754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.048037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.048308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.049057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.050720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.068357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 943/1890] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:49:53.069846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.086163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba3d9-7f18-4ae9-aab2-4a438c7db772 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.092624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.094323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.094847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.095166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] PATH_INFO: `/v3/projects/1b05224f404f4f018926be2d90241768/users/8a8adacdeaeb4a639b4a6204817dcd74/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.097320 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b05224f404f4f018926be2d90241768, user_id=8a8adacdeaeb4a639b4a6204817dcd74, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.100788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a51cd8-f9c8-427e-8d23-3b361d29cc35 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.101852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.102658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.103041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.103676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.104579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.107552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.115328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 944/1891] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:53 2022] PUT /identity/v3/projects/1b05224f404f4f018926be2d90241768/users/8a8adacdeaeb4a639b4a6204817dcd74/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:53.124267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 948/1892] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:53.126885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.127969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.128250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.129274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.143478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:53.145927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b1c58-04b1-47e9-93f7-a307114456d8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.161956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] MFA Rules not processed for user `8a8adacdeaeb4a639b4a6204817dcd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:53.163371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:53.228423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.230318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.230788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.231068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] PATH_INFO: `/v3/projects/63d5a24a95f344fa9589775a5c756759/users/9abc613fc4f842a5a53d840e3e9e9ad1/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.234339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d5a24a95f344fa9589775a5c756759, user_id=9abc613fc4f842a5a53d840e3e9e9ad1, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.238423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde1ca1-ff8a-466e-8c0c-07987315f586 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.243126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:53.254581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 949/1893] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:53 2022] PUT /identity/v3/projects/63d5a24a95f344fa9589775a5c756759/users/9abc613fc4f842a5a53d840e3e9e9ad1/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:53.266241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03330884-9070-4753-b29a-94608fed9c27 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.266532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03330884-9070-4753-b29a-94608fed9c27 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.266813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03330884-9070-4753-b29a-94608fed9c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.282542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 945/1894] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:53.290548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.301305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:53.348241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-03330884-9070-4753-b29a-94608fed9c27 None None] MFA Rules not processed for user `9abc613fc4f842a5a53d840e3e9e9ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:53.350489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-03330884-9070-4753-b29a-94608fed9c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:53.370503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6985437-2e38-49d7-a641-f291c16b7fd0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.429870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.430672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.430960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.431154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.438313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 950/1895] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:53.447386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03330884-9070-4753-b29a-94608fed9c27 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.449005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-03330884-9070-4753-b29a-94608fed9c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:53.461720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03330884-9070-4753-b29a-94608fed9c27 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.465448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.466146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.466374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.466651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.477568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.479007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.490166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 951/1896] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:53.507702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.509168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.513110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:53.519914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 946/1897] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:53.692151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.700930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d14775-207e-48b9-80b8-eca9d6cedff9 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.704805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.705476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.705748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.706041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.718347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:53.727597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bc717-4839-4e9c-8365-a7255a8c91e1 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:53.731847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:53.732150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:53.732439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:53.733292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:53.785991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.788358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.799546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 952/1898] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:53.802361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:53.804805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd5e1-0f33-455b-bec1-00dd31760df0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:53.819209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 947/1899] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:54.405717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:54.490166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c3018-25b8-4d39-8435-f766ac8fb1a1 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:54.499211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:54.500123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:54.500461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:54.500747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] PATH_INFO: `/v3/users/126305ecdeab43cfabf009d2d93a7bb3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:54.502076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126305ecdeab43cfabf009d2d93a7bb3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:54.513627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:54.525452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:54.740143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126305ecdeab43cfabf009d2d93a7bb3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:54.757470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126305ecdeab43cfabf009d2d93a7bb3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:54.758389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126305ecdeab43cfabf009d2d93a7bb3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:54.759573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126305ecdeab43cfabf009d2d93a7bb3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:54.771491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126305ecdeab43cfabf009d2d93a7bb3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:54.849519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 953/1900] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:54 2022] DELETE /identity/v3/users/126305ecdeab43cfabf009d2d93a7bb3 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:54.854275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:54.854769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:54.854989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:54.875680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:54.876241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:54.958241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 948/1901] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:54.964869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:54.966384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:55.025537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38e8b3-e28f-4114-b19a-7e6aa6a617a9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:55.029132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:55.030432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:55.030794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:55.031187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:55.041309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:55.042773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:55.055860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 954/1902] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:49:56.083481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:56.085331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:56.121411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f163bc6-01e8-461f-850f-959e8a8fc06c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:56.133327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:56.134353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:56.134622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:56.135077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] PATH_INFO: `/v3/projects/f5d55db2d92941c1833c93cd80acd336` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:56.136404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d55db2d92941c1833c93cd80acd336)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:56.138542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:56.277483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d55db2d92941c1833c93cd80acd336', 'request_id': 'req-88014b0b-9191-4c1e-825d-f554c2ec6901'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.286605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d55db2d92941c1833c93cd80acd336', 'request_id': 'req-88014b0b-9191-4c1e-825d-f554c2ec6901'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.335314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d55db2d92941c1833c93cd80acd336', 'request_id': 'req-88014b0b-9191-4c1e-825d-f554c2ec6901'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.348099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] The token cache is being invalidate because project f5d55db2d92941c1833c93cd80acd336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:56.348420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.349106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.350689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 949/1903] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:56 2022] DELETE /identity/v3/projects/f5d55db2d92941c1833c93cd80acd336 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:56.354876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:56.366008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7deead4-15be-415f-ba1b-84f7b74fb675 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:56.433132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:56.433928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:56.434218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:56.434477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] PATH_INFO: `/v3/users/8a8adacdeaeb4a639b4a6204817dcd74` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:56.435710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8adacdeaeb4a639b4a6204817dcd74)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:56.438101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:56.622728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:56.644362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8adacdeaeb4a639b4a6204817dcd74'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.658274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8adacdeaeb4a639b4a6204817dcd74'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.659061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8adacdeaeb4a639b4a6204817dcd74'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.659719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8adacdeaeb4a639b4a6204817dcd74'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.666516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8adacdeaeb4a639b4a6204817dcd74'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:56.784714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 955/1904] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:56 2022] DELETE /identity/v3/users/8a8adacdeaeb4a639b4a6204817dcd74 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:57.297502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:57.300366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:57.334088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88014b0b-9191-4c1e-825d-f554c2ec6901 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:57.355971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:57.437676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:57.438580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.439229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.439606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] PATH_INFO: `/v3/projects/1b05224f404f4f018926be2d90241768` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.441455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b05224f404f4f018926be2d90241768)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:57.444326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:57.583581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:57.585103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:57.590579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b05224f404f4f018926be2d90241768', 'request_id': 'req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.602107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678ba80-c767-4bd3-9748-9ed908a598f4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:57.604339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b05224f404f4f018926be2d90241768', 'request_id': 'req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.683625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:57.684398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b05224f404f4f018926be2d90241768', 'request_id': 'req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.685071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.685183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.685846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] PATH_INFO: `/v3/users/4d55ff81b28443a9a200488a089a4f5e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.687194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d55ff81b28443a9a200488a089a4f5e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:57.689393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:57.710531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] The token cache is being invalidate because project 1b05224f404f4f018926be2d90241768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:57.713594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.713594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-74ecfdfd-6ea3-43d6-82e8-9cdace04afb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.714328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 950/1905] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:57 2022] DELETE /identity/v3/projects/1b05224f404f4f018926be2d90241768 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:57.720662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d812-5d34-4574-a02c-0a9788f411f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.724885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d812-5d34-4574-a02c-0a9788f411f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.725210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59d812-5d34-4574-a02c-0a9788f411f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.745077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ea59d812-5d34-4574-a02c-0a9788f411f4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:57.746256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ea59d812-5d34-4574-a02c-0a9788f411f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:57.755969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 951/1906] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:57.765717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.766007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.766286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.785724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:57.786993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:57.800004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 952/1907] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:57.801365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:57.811699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e58c4-0957-4102-94d1-a3ea4b54004b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:57.817066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:57.819256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.819554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.819838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.821215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:49:57.821535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:57.823544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:57.841331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 953/1908] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:49:57.844782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:57.856188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e6fc-3e2b-4970-b616-d0802573ef35 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:57.861256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:57.861923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:57.862207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:57.862485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:57.863349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:57.865294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:57.896505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d55ff81b28443a9a200488a089a4f5e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.911530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d55ff81b28443a9a200488a089a4f5e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.912727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d55ff81b28443a9a200488a089a4f5e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.913359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d55ff81b28443a9a200488a089a4f5e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.921190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3509194b-c20a-4e19-a974-c87388d4e724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d55ff81b28443a9a200488a089a4f5e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:57.976001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 954/1909] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:49:57 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:57.980054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:57.981644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.021193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 956/1910] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:57 2022] DELETE /identity/v3/users/4d55ff81b28443a9a200488a089a4f5e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:58.025383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.025675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.025955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.051104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:58.052871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.070257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceedb3-1712-4b95-898e-6cff07b4114d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.079057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:58.081010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:58.138320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 957/1911] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:58.148175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.148859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.149189 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.149469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.150021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.151157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.151893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.153585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.230811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837450ac-1e99-420d-8df5-7f7a63721a19 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.234588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.235227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.235512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.235783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.245014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.246457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.251083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 955/1912] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:49:57 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:49:58.255177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.257055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 958/1913] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:58.275673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a98e-36c3-40b0-be09-d7c3610164b9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.282112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.283053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.283246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.283528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.284261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.286307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.299787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 956/1914] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:58.301067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.312658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a665-a34d-4160-bc86-747859a567d8 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.319347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.320282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.320635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.320979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.321946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.323481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff662ab7-9372-490e-bb8e-875b527faf35 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.338318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 959/1915] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:49:58.340022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.351351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad39dd4-05e5-45ff-b1e9-8fb526653417 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.357602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.363074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.363074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.363074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] PATH_INFO: `/v3/projects/071241189b764364a50d292b61fac2f0/users/4ab5acaa36b34f3db3e0443c6d524db0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.378294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=071241189b764364a50d292b61fac2f0, user_id=4ab5acaa36b34f3db3e0443c6d524db0, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.384576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.483146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 957/1916] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:49:58 2022] PUT /identity/v3/projects/071241189b764364a50d292b61fac2f0/users/4ab5acaa36b34f3db3e0443c6d524db0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:58.493873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.494168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.494449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.548099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] MFA Rules not processed for user `4ab5acaa36b34f3db3e0443c6d524db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:58.549456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.687189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 960/1917] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:49:58.746283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.748983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.808118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b68f-9be3-47c9-af3c-cbc74f0257bf admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.813737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.814675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.815245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.815669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.816334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.817860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:58.829100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.829656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.841403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 958/1918] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:49:58.863277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:58.895292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:58.912743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f7018-8d55-49d4-bbf1-9133ce9fa4c1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.918908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.919368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.919660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.919974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] PATH_INFO: `/v3/projects/8952fffef8804b3590800265465bd60e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.921460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8952fffef8804b3590800265465bd60e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.924204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.942103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada4fb-212a-41d8-8e47-c197ca2455e6 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:58.947129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:58.948000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:58.948322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:58.948600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] PATH_INFO: `/v3/users/4ab5acaa36b34f3db3e0443c6d524db0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:58.949817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab5acaa36b34f3db3e0443c6d524db0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:58.952421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:58.960553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:59.066880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8952fffef8804b3590800265465bd60e', 'request_id': 'req-a6fd0d2c-8631-4d1d-a885-7569484b6064'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.073615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8952fffef8804b3590800265465bd60e', 'request_id': 'req-a6fd0d2c-8631-4d1d-a885-7569484b6064'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.161456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8952fffef8804b3590800265465bd60e', 'request_id': 'req-a6fd0d2c-8631-4d1d-a885-7569484b6064'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.178455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] The token cache is being invalidate because project 8952fffef8804b3590800265465bd60e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:49:59.178455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.179756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.182183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 961/1919] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:49:58 2022] DELETE /identity/v3/projects/8952fffef8804b3590800265465bd60e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:59.188187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:59.190013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:59.213584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab5acaa36b34f3db3e0443c6d524db0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.225773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab5acaa36b34f3db3e0443c6d524db0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.243235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab5acaa36b34f3db3e0443c6d524db0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.243987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab5acaa36b34f3db3e0443c6d524db0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.274535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd0d2c-8631-4d1d-a885-7569484b6064 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:59.279643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:59.280805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:59.281121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:59.281394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] PATH_INFO: `/v3/users/9abc613fc4f842a5a53d840e3e9e9ad1` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:59.282600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abc613fc4f842a5a53d840e3e9e9ad1)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:59.284628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:59.367286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fbcce039-6f3d-492b-b128-53432f5b9a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab5acaa36b34f3db3e0443c6d524db0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.374115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 959/1920] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:58 2022] DELETE /identity/v3/users/4ab5acaa36b34f3db3e0443c6d524db0 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:49:59.378518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:59.384476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:59.384476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:59.397591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:49:59.399021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:59.481782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 960/1921] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:49:59.485386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:49:59.487231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:59.492096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abc613fc4f842a5a53d840e3e9e9ad1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.511515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abc613fc4f842a5a53d840e3e9e9ad1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.513080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abc613fc4f842a5a53d840e3e9e9ad1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.513854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abc613fc4f842a5a53d840e3e9e9ad1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.525151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abc613fc4f842a5a53d840e3e9e9ad1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:49:59.610384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:59.620737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 962/1922] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:49:59 2022] DELETE /identity/v3/users/9abc613fc4f842a5a53d840e3e9e9ad1 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:49:59.626169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318ec27-1826-4b98-805e-58a64f8f92cb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:49:59.674429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:49:59.685505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:49:59.686175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:49:59.686549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:49:59.686824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:49:59.688762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:49:59.779069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:49:59.782298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c7d5b-6a6e-4330-8513-f713b0a80f2f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:49:59.796080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 961/1923] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:50:00.477869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:00.479458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:00.500466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb85-265d-415a-bec2-1e84846a25f7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:00.506642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:00.507754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:00.507948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:00.508320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] PATH_INFO: `/v3/projects/63d5a24a95f344fa9589775a5c756759` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:00.509821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d5a24a95f344fa9589775a5c756759)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:00.511896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:00.684826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d5a24a95f344fa9589775a5c756759', 'request_id': 'req-ab48af8c-ac71-4d8e-b686-ab7071983020'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:00.698705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d5a24a95f344fa9589775a5c756759', 'request_id': 'req-ab48af8c-ac71-4d8e-b686-ab7071983020'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:00.813383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d5a24a95f344fa9589775a5c756759', 'request_id': 'req-ab48af8c-ac71-4d8e-b686-ab7071983020'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:00.840379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] The token cache is being invalidate because project 63d5a24a95f344fa9589775a5c756759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:00.840379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:00.847569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:00.849307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 963/1924] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:00 2022] DELETE /identity/v3/projects/63d5a24a95f344fa9589775a5c756759 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:00.854613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3ebb3-1329-4909-b7b9-f46243a68736 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:00.855228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3ebb3-1329-4909-b7b9-f46243a68736 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:00.855324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3ebb3-1329-4909-b7b9-f46243a68736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:00.869159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:00.872940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:00.901843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b8f3ebb3-1329-4909-b7b9-f46243a68736 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:00.903720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b8f3ebb3-1329-4909-b7b9-f46243a68736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:00.983788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48af8c-ac71-4d8e-b686-ab7071983020 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:00.987387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 962/1925] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:00.995897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:00.996291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:00.996575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:00.997523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:00.998994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:00.999773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.000053 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] PATH_INFO: `/v3/projects/071241189b764364a50d292b61fac2f0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.001684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071241189b764364a50d292b61fac2f0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.003619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.018860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:01.019900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:01.029606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 963/1926] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:01.033835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.048508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785d56-0906-45a1-998c-55649d89d36a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.053347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.054818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.054818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.054818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.055780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:01.056083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.057782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.070905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 964/1927] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:50:01.075083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.085907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27387f5e-6364-45f2-996d-ac3b0de95963 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.091290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.092718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.093003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.093315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.094010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.096055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.115646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071241189b764364a50d292b61fac2f0', 'request_id': 'req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:01.122854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071241189b764364a50d292b61fac2f0', 'request_id': 'req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:01.187222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 965/1928] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:01.191021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.202352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071241189b764364a50d292b61fac2f0', 'request_id': 'req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:01.215338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] The token cache is being invalidate because project 071241189b764364a50d292b61fac2f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:01.217323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:01.217681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-eb7dd3f3-624a-4ac5-9962-032c07c5cab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:01.219642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 964/1929] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:00 2022] DELETE /identity/v3/projects/071241189b764364a50d292b61fac2f0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:01.224524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3baf-af13-40c9-b21f-ac23fbeca316 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.224950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3baf-af13-40c9-b21f-ac23fbeca316 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.225499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3baf-af13-40c9-b21f-ac23fbeca316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.246199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c2ac3baf-af13-40c9-b21f-ac23fbeca316 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:01.247724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c2ac3baf-af13-40c9-b21f-ac23fbeca316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:01.257872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:01.303874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:01.313225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5910d-bfe6-4022-bf33-54e0c723e588 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.319867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.321417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.321582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.321956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.323419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.325184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.352880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 965/1930] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:01.354904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.355189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.355468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.378735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:01.390312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:01.390312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 966/1931] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:01.393134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.402945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba123e0-cbbf-4b16-b95c-bcfd33cee8bf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.407820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.408482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.409259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.409259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.409455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 966/1932] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:50:01.409790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:01.410175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.412382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.413695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.415317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:01.436851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276708cd-eb30-44c2-a4b9-cad746051891 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.440213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 967/1933] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:50:01.449239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.449360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.449360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.449360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.449360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.449681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.449681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.460903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672413e-7eec-41d0-8ba8-007f2c5440ca admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.463852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 967/1934] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:01.467808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.468854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.469634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.469938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.470220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.471067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.473203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.478787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956bafd-8d00-49a2-9741-052f0f97e07d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.485219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.490424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.490424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.490424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.490424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.490424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.500521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 968/1935] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:01.504622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.515395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5f590-edba-418a-898d-d93c5c002106 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.520263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.521239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.524804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.524804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] PATH_INFO: `/v3/projects/9195c8ce0ce54fb79917c5c2a8627a7b/users/4940de9ddc574406b7ec80075762a7dc/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.539293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9195c8ce0ce54fb79917c5c2a8627a7b, user_id=4940de9ddc574406b7ec80075762a7dc, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.542081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc22d-1f7a-4534-bebf-d40a81ddead9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.582458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 968/1936] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:01.588484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.644273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 969/1937] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:01 2022] PUT /identity/v3/projects/9195c8ce0ce54fb79917c5c2a8627a7b/users/4940de9ddc574406b7ec80075762a7dc/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:01.649975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.650385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.650733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.663668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ddf6-ebe8-4753-bd31-37851fcfbe4c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.679113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] MFA Rules not processed for user `4940de9ddc574406b7ec80075762a7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:01.680900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:01.730915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.731597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.731876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.732184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.732882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.734836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.757085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 970/1938] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:01.761943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.773567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7ba08-19ce-44f0-a8d0-a0ce7e93d402 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.778214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.778942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.779392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.779676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.780382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.782303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.819496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 969/1939] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:01.823699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.833657 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503816f-af5e-43d3-8a85-280ae2937ea4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.838127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.838765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.839045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.839319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.840072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.841838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.843130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:01.852103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 970/1940] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:01.855755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.865772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d885f0-7b61-4fab-a141-3b5f792a1e11 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.870341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.879196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.879196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.879196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.879196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.879196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.885283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 971/1941] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:01.889400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.899138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cfc39-84f1-4325-8712-27ddc83a6664 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.904487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.905625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.907994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.907994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] PATH_INFO: `/v3/projects/132540ab920b47bb84b119446bbd0251/users/b84d6897ac424cbfbb505ec024c29bff/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.908202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 971/1942] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:01.910760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:01.925828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132540ab920b47bb84b119446bbd0251, user_id=b84d6897ac424cbfbb505ec024c29bff, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.930396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91ba0e-b924-48ab-b2ae-e52f382de9b6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:01.985649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87daa-5b89-40d8-b8cf-ea01ce7a0535 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:01.991033 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:01.997696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:01.997696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:01.997696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:01.997696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:01.997696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.005734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:02.026221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 972/1943] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:01 2022] PUT /identity/v3/projects/132540ab920b47bb84b119446bbd0251/users/b84d6897ac424cbfbb505ec024c29bff/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:02.041292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.041292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.041292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.070002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] MFA Rules not processed for user `b84d6897ac424cbfbb505ec024c29bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:02.071391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:02.095776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 972/1944] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:01 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:02.100869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.114022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf242151-6bd8-4041-9d24-8765bdf3de1d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.149083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 973/1945] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:02.152919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.183492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.240899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 973/1946] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:02.242079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.242079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c209c-6728-4e61-8fac-8d15a3637cf4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.242079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.242368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4482fc-598c-40ca-8e55-9142ae0a0cf0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.242476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.242476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.242476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] PATH_INFO: `/v3/projects/38e995173ed9423195e0d129f8454b64/users/849c9c6058b14fe8a32fedb6c053bf97/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.242725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.242725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.242725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.242725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.243058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.243058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.250590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e995173ed9423195e0d129f8454b64, user_id=849c9c6058b14fe8a32fedb6c053bf97, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.253389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.305725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 974/1947] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:02.306693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.316358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 974/1948] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:02 2022] PUT /identity/v3/projects/38e995173ed9423195e0d129f8454b64/users/849c9c6058b14fe8a32fedb6c053bf97/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:02.324345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.360064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:02.387558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324333f-3cb7-4b9a-a6c4-cfbf70a611eb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.400768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c6e75-40bf-406a-81f8-3edfdaafe4c5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.408834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.409560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.409936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.410291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.411104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.413706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.427777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 975/1949] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:02.432115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.455532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a001c85-babc-4a36-8063-a7bc957ea814 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.460243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.461166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.461721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.462011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] PATH_INFO: `/v3/projects/38e995173ed9423195e0d129f8454b64/users/849c9c6058b14fe8a32fedb6c053bf97/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.464134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e995173ed9423195e0d129f8454b64, user_id=849c9c6058b14fe8a32fedb6c053bf97, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.467456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf11fb-ef60-4155-a2e6-47c5fafdce2a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.475083 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.475834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.476186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.476526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.477328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.478614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 976/1950] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:02 2022] PUT /identity/v3/projects/38e995173ed9423195e0d129f8454b64/users/849c9c6058b14fe8a32fedb6c053bf97/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:02.479669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.489083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.489807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.489807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.517707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] MFA Rules not processed for user `849c9c6058b14fe8a32fedb6c053bf97`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:02.518918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:02.552269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 975/1951] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:50:02.556556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.560741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:02.566872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2fddb-d76a-4270-b2b1-825d6c7616b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.611014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 977/1952] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:02.618312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.620829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:02.633944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.634732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.635112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.635866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.636467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.639063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.651838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 976/1953] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:02.657356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.670629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970edc-567c-45de-8f00-2710bfa0f0bb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.676756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.677684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.678061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.678429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.679259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.681797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.702707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 977/1954] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:02.704789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:02.706166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542b375-9899-4e0f-b113-3089009ba9c4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.710322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.711394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.711675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.711949 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.717580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3fb56-28a3-4722-9d39-f03cba8341e9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:02.722812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:02.723628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.723915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.724197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] PATH_INFO: `/v3/projects/a60bd2ab99d248df81f2c0568e2d17fb/users/4e09036b760f46aab96759d15a3c11d1/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.743970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60bd2ab99d248df81f2c0568e2d17fb, user_id=4e09036b760f46aab96759d15a3c11d1, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.746672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.790119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:02.796667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf87b6-1d36-48af-8748-6f3195d4dd44 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:02.802800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 978/1955] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:50:02.819170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 978/1956] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:02 2022] PUT /identity/v3/projects/a60bd2ab99d248df81f2c0568e2d17fb/users/4e09036b760f46aab96759d15a3c11d1/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:02.830613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:02.831184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:02.831700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:02.872829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] MFA Rules not processed for user `4e09036b760f46aab96759d15a3c11d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:02.875342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:02.971507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 979/1957] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:02.975953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.019005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:03.062163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caacaf-e018-4bc2-8428-82dfc06ba2f0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.069336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.070620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.071023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.071404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.073181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.077031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.180983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 979/1958] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:02 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:03.184633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.215247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdaab3-c1e3-409d-bd03-b0eebb51533a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.222762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:03.287604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.288279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.288560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.288832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.289569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.291604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.394735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 980/1959] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:03 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:03.395674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.406202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435d3fd-58be-4194-9854-1b76fc8624a4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.410711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.411394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.412590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.416381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.416381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.416381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.448479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 980/1960] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:03.449526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.461432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ed9d-35f3-4033-800b-1151ae86cb1d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.470941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.474421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.474421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.474421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] PATH_INFO: `/v3/projects/2e2602783b8a4c20a292f9912457a2d7/users/498dde220b204c9db9b58e6b5758985f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.500217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2602783b8a4c20a292f9912457a2d7, user_id=498dde220b204c9db9b58e6b5758985f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.503229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.588981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 981/1961] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:03 2022] PUT /identity/v3/projects/2e2602783b8a4c20a292f9912457a2d7/users/498dde220b204c9db9b58e6b5758985f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:03.593299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.605098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eeb2e3-2c71-45cd-bdda-9646e5271e97 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.670708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.671532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.671664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.671915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.672652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.674354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2398a-5845-49ab-b45c-996a19c6b151 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.691262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 981/1962] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:03.691355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.700650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beecfd2-6b97-498f-8b91-03a4309f5942 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.705473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.706371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.706637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.706903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] PATH_INFO: `/v3/projects/2e2602783b8a4c20a292f9912457a2d7/users/498dde220b204c9db9b58e6b5758985f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.710159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2602783b8a4c20a292f9912457a2d7, user_id=498dde220b204c9db9b58e6b5758985f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.712859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.723915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 982/1963] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:03 2022] PUT /identity/v3/projects/2e2602783b8a4c20a292f9912457a2d7/users/498dde220b204c9db9b58e6b5758985f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:03.736790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.737110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.737403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.766167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] MFA Rules not processed for user `498dde220b204c9db9b58e6b5758985f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:03.767410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:03.849108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 982/1964] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:03.856497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:03.858979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:03.919184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a9cc1-0e7a-4061-b6b6-5c025fd1b223 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:03.924008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:03.924653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:03.924932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:03.925233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:03.983760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:03.985286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f6e06-84af-4398-a8fe-b49058250608 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:03.994032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 983/1965] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:06.237817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:06.239974 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:06.278719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:06.312962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052131a-6835-4b3c-893e-fc4e1ab8d746 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:06.317968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:06.319757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:06.319757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:06.319757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] PATH_INFO: `/v3/users/fb8798dc5940426497a44fbfd85a410b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:06.320675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8798dc5940426497a44fbfd85a410b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:06.322632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:06.552966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8798dc5940426497a44fbfd85a410b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:06.565980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8798dc5940426497a44fbfd85a410b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:06.571868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8798dc5940426497a44fbfd85a410b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:06.571868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8798dc5940426497a44fbfd85a410b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:06.574209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8798dc5940426497a44fbfd85a410b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:06.652907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 983/1966] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:06 2022] DELETE /identity/v3/users/fb8798dc5940426497a44fbfd85a410b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:06.657575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:06.658088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:06.658188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:06.676764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:06.678131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:06.696826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:06.742821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 984/1967] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:06.752610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:06.754318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:06.814877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7342d-1477-480b-8daf-e91a3d241275 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:06.818536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:06.819188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:06.819595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:06.819733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:06.828851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:06.830299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:06.840102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 984/1968] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:07.008840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:07.010276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:07.072457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0751f5-2d89-4a53-9748-27ff78f74fd2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:07.076169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:07.079208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.079208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.079208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.079540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:07.142720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:07.144373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a3cf-bb86-473a-ba15-a2a5081a95a0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:07.156292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 985/1969] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:07.285818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:07.287419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:07.300699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9411575-5d19-48a3-af35-c8341cfe8824 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:07.307143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:07.307949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.308233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.308509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] PATH_INFO: `/v3/projects/d61b15bdc1bd42449ae37abc750e2e9d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.309790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d61b15bdc1bd42449ae37abc750e2e9d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:07.311711 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:07.454086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:07.456904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd61b15bdc1bd42449ae37abc750e2e9d', 'request_id': 'req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:07.463223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd61b15bdc1bd42449ae37abc750e2e9d', 'request_id': 'req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:07.538665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd61b15bdc1bd42449ae37abc750e2e9d', 'request_id': 'req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:07.551326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] The token cache is being invalidate because project d61b15bdc1bd42449ae37abc750e2e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:07.551782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:07.552632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ec95952-72d3-4c0f-8cca-0c3a2a5032d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:07.555012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 985/1970] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:07 2022] DELETE /identity/v3/projects/d61b15bdc1bd42449ae37abc750e2e9d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:07.560213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.560928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.561321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.581207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:07.582423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:07.653346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 986/1971] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:07.661440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.661736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.662019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.683143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:07.684387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:07.693396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 986/1972] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:07.697864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:07.708948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a965-a09d-405b-90a0-a41730663377 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:07.714503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:07.714503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.714861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.715251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.716314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:07.716314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:07.717985 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:07.731381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 987/1973] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:50:07.737122 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:07.748655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c1c2-ee8b-461a-bcc4-fb3c3e62f3ff None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:07.754078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:07.754961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.760786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.760786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.760786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:07.760786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:07.868604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 987/1974] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:07 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:07.872892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:07.880442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:07.884488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846864-5f42-4e77-9f8e-75b0d4d14342 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:07.951410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:07.952408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:07.952686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:07.952955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:07.953679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:07.956302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.030393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 988/1975] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:07 2022] POST /identity/v3/users => generated 278 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:08.035109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.046348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9697c9-f91c-40a4-9141-a032f0c69a3b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.051092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.051791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.052073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.052348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.053120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:08.055275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.066914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 988/1976] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:08.071076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.083302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c551ffe-2fb6-4cae-b8bd-ca150bb7d6f2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.089451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.090167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.090448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.090763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.091572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:08.093420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f901a-fa6c-4ae0-b4ef-74daa792341c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.103916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 989/1977] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:08.108855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.118855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9ce12-84da-42b0-99fa-5d1e151cca6d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.124488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.125400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.125675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.125949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] PATH_INFO: `/v3/projects/abe63999625c4d5cb90301c21fe70bbf/users/71d8c269d7594cee961398806e946084/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.141651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe63999625c4d5cb90301c21fe70bbf, user_id=71d8c269d7594cee961398806e946084, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:08.144300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.202500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 989/1978] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:08 2022] PUT /identity/v3/projects/abe63999625c4d5cb90301c21fe70bbf/users/71d8c269d7594cee961398806e946084/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:08.212996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.217806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.217806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.241361 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] MFA Rules not processed for user `71d8c269d7594cee961398806e946084`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:08.242643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.307303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 990/1979] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:08.338494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.339853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.398549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608f3f9-08e8-48f1-a914-0b02fd96a1cd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.402687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.403359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.403663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.404436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.413271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:08.414747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdd6af-9bda-466c-b514-28b526cb90e0 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.424842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 990/1980] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:08 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:08.439288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.510174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69c25b-3d02-417f-bbd5-15a81ea9741d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.514919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.515793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.516080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.516354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] PATH_INFO: `/v3/users/71d8c269d7594cee961398806e946084` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.517579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71d8c269d7594cee961398806e946084)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:08.519506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:08.533842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:08.729217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71d8c269d7594cee961398806e946084'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:08.736524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71d8c269d7594cee961398806e946084'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:08.748913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71d8c269d7594cee961398806e946084'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:08.749670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71d8c269d7594cee961398806e946084'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:08.817881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71d8c269d7594cee961398806e946084'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:08.824511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 991/1981] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:08 2022] DELETE /identity/v3/users/71d8c269d7594cee961398806e946084 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:08.829953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.830248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.830532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.848866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:08.850400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.878144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.879666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.918204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 991/1982] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:08.928272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:08.929953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.940029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46518-c639-438e-a749-0532df09d784 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.944670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.945549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.945839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.946267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:08.948105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:08.948779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:08.991600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56195623-f7ad-4822-9abb-0888ffe76ac1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:08.995596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:08.996433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:08.996889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:08.997252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.009348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:09.010932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:09.020725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:09.022197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 992/1983] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:50:09.022371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034887f1-1d0d-4581-9593-ad613ed0d16e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:09.034623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 992/1984] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:50:09.507456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:09.508968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:09.522379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff287b-698d-4904-91cf-608e171bd464 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:09.527161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:09.528027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:09.528314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:09.528581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] PATH_INFO: `/v3/projects/abe63999625c4d5cb90301c21fe70bbf` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.529830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe63999625c4d5cb90301c21fe70bbf)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:09.532166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:09.653370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe63999625c4d5cb90301c21fe70bbf', 'request_id': 'req-1455c908-31cb-4440-9d35-145a61333726'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:09.660341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe63999625c4d5cb90301c21fe70bbf', 'request_id': 'req-1455c908-31cb-4440-9d35-145a61333726'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:09.714895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe63999625c4d5cb90301c21fe70bbf', 'request_id': 'req-1455c908-31cb-4440-9d35-145a61333726'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:09.725758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] The token cache is being invalidate because project abe63999625c4d5cb90301c21fe70bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:09.726069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:09.726696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1455c908-31cb-4440-9d35-145a61333726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:09.728242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 993/1985] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:09 2022] DELETE /identity/v3/projects/abe63999625c4d5cb90301c21fe70bbf => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:09.734673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:09.735148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:09.735375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.753829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:09.754946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:09.793871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:09.815894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 993/1986] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:09.823656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:09.823946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:09.824250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.842984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:09.844170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:09.851950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 994/1987] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:09.856100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:09.866275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2643b-2165-4e9b-9fe9-5bb2c9361b68 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:09.871359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:09.872001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:09.872283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:09.872556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.873310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:09.873613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:09.876393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:09.888059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 994/1988] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:50:09.892916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:09.902063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb48cd-ad66-4b84-82d7-7e0524dfc691 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:09.907081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:09.907724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:09.908003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:09.908274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:09.908957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:09.910866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.020776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 995/1989] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:09 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:10.026459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:10.036769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109ff0d-beda-4117-97b8-80717592d2f1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:10.092740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:10.093459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.093747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.094019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.094712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:10.096615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.183762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 995/1990] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:10 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:10.185088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:10.197498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06837306-d7fa-4634-a503-3870f63784e0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:10.202259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:10.202956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.203238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.203516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.204256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:10.205909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.218579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 996/1991] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:10.222362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:10.236967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60ecc6-047c-4a36-a74a-c88e869ddc71 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:10.241356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:10.242052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.242337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.242613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.243340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:10.244930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1bb7d-d863-450f-ac09-7e0c17eb4e3a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.256144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 996/1992] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:10.260256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:10.276545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f873-afab-4de0-8974-fecbd9dacd9a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:10.276545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:10.276938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.277249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.277529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] PATH_INFO: `/v3/projects/48c727a7e3454f26a810b5bf457023f9/users/b3a84a70c193447d8c5ef91aad92a449/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.293671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c727a7e3454f26a810b5bf457023f9, user_id=b3a84a70c193447d8c5ef91aad92a449, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:10.296904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.355740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 997/1993] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:10 2022] PUT /identity/v3/projects/48c727a7e3454f26a810b5bf457023f9/users/b3a84a70c193447d8c5ef91aad92a449/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:10.366332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.366621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.366898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.393669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] MFA Rules not processed for user `b3a84a70c193447d8c5ef91aad92a449`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:10.394863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:10.458720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 997/1994] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:10.470427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:10.472133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:10.511824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:10.528587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0035ee-0c7b-4dcc-a27b-46ba7ae73383 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:10.532958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:10.533634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:10.533918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:10.534197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:10.542532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:10.543992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:10.554394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 998/1995] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:12.302003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:12.303827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:12.315759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4f19f-343c-4ef5-a87f-76ccaad39c44 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:12.319284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:12.319915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:12.320198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:12.320471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:12.322165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:12.357735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:12.386058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:12.387494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d3fbc-f045-43c7-8a43-f00717e7b522 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:12.397773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 998/1996] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:20.612373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:21.194387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:21.195826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:21.264925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847922a-487f-4a5f-b5e1-d14a549051ce service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:21.270103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:21.270973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:21.271253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:21.271525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] PATH_INFO: `/v3/users/b84d6897ac424cbfbb505ec024c29bff` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:21.272701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84d6897ac424cbfbb505ec024c29bff)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:21.274648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:21.410421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84d6897ac424cbfbb505ec024c29bff'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:21.433617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84d6897ac424cbfbb505ec024c29bff'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:21.438104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84d6897ac424cbfbb505ec024c29bff'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:21.441776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84d6897ac424cbfbb505ec024c29bff'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:21.452125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84d6897ac424cbfbb505ec024c29bff'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:21.538101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 999/1997] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:21 2022] DELETE /identity/v3/users/b84d6897ac424cbfbb505ec024c29bff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:21.542535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:21.542979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:21.543096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:21.579459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:21.584495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:21.674337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 999/1998] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:21.681314 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:21.683225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:21.757944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e74884-4e0a-474f-b61a-202f7ec3cf95 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:21.761902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:21.762633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:21.762922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:21.763291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:21.775161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:21.784398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:21.791667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1000/1999] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:22.266630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:22.268219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:22.280329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e099310-14fe-4a90-94f6-7b77583199d1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:22.285941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:22.286851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:22.287142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:22.287416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] PATH_INFO: `/v3/projects/132540ab920b47bb84b119446bbd0251` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:22.288617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=132540ab920b47bb84b119446bbd0251)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:22.296707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:22.423597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132540ab920b47bb84b119446bbd0251', 'request_id': 'req-eb71128e-0f2e-4470-8807-1ccb09ff77bc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.431032 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132540ab920b47bb84b119446bbd0251', 'request_id': 'req-eb71128e-0f2e-4470-8807-1ccb09ff77bc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.533327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132540ab920b47bb84b119446bbd0251', 'request_id': 'req-eb71128e-0f2e-4470-8807-1ccb09ff77bc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.545206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] The token cache is being invalidate because project 132540ab920b47bb84b119446bbd0251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:22.545830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.546366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.548983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1000/2000] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:22 2022] DELETE /identity/v3/projects/132540ab920b47bb84b119446bbd0251 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:22.553434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:22.563809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d9b43-a96c-4816-ac65-9541cffa784a service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:22.591018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:22.650327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:22.651452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:22.651805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:22.652155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] PATH_INFO: `/v3/users/4e09036b760f46aab96759d15a3c11d1` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:22.653537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e09036b760f46aab96759d15a3c11d1)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:22.655553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:22.836903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e09036b760f46aab96759d15a3c11d1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.849838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e09036b760f46aab96759d15a3c11d1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.850502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e09036b760f46aab96759d15a3c11d1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.851851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e09036b760f46aab96759d15a3c11d1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.859858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e09036b760f46aab96759d15a3c11d1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:22.932444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1001/2001] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:22 2022] DELETE /identity/v3/users/4e09036b760f46aab96759d15a3c11d1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:23.350862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:23.352438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:23.365828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71128e-0f2e-4470-8807-1ccb09ff77bc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:23.417675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:23.424754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:23.425633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:23.425914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:23.426187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] PATH_INFO: `/v3/projects/a60bd2ab99d248df81f2c0568e2d17fb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:23.427339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60bd2ab99d248df81f2c0568e2d17fb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:23.429509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:23.527666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60bd2ab99d248df81f2c0568e2d17fb', 'request_id': 'req-6685b190-ae2d-46f1-bd08-114ac8fe52b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.534311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60bd2ab99d248df81f2c0568e2d17fb', 'request_id': 'req-6685b190-ae2d-46f1-bd08-114ac8fe52b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.600645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60bd2ab99d248df81f2c0568e2d17fb', 'request_id': 'req-6685b190-ae2d-46f1-bd08-114ac8fe52b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.612124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] The token cache is being invalidate because project a60bd2ab99d248df81f2c0568e2d17fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:23.612408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.613092 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.614698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1001/2002] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:23 2022] DELETE /identity/v3/projects/a60bd2ab99d248df81f2c0568e2d17fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:23.618708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:23.620220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:23.633931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902456aa-44c8-4055-bc51-f4b9bd842542 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:23.690647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:23.691514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:23.691793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:23.692066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] PATH_INFO: `/v3/users/498dde220b204c9db9b58e6b5758985f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:23.693273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=498dde220b204c9db9b58e6b5758985f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:23.695174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:23.840957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:23.843485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498dde220b204c9db9b58e6b5758985f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.855236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498dde220b204c9db9b58e6b5758985f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.855910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498dde220b204c9db9b58e6b5758985f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.856620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498dde220b204c9db9b58e6b5758985f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.864154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4035c90c-3c14-4d39-815e-3074bd957dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498dde220b204c9db9b58e6b5758985f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:23.960463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1002/2003] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:23 2022] DELETE /identity/v3/users/498dde220b204c9db9b58e6b5758985f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:24.364570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:24.366131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:24.380585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685b190-ae2d-46f1-bd08-114ac8fe52b6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:24.445352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:24.446215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:24.446496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:24.446770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] PATH_INFO: `/v3/projects/2e2602783b8a4c20a292f9912457a2d7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:24.447940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2602783b8a4c20a292f9912457a2d7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:24.449868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:24.578701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2602783b8a4c20a292f9912457a2d7', 'request_id': 'req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:24.584604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2602783b8a4c20a292f9912457a2d7', 'request_id': 'req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:24.658254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2602783b8a4c20a292f9912457a2d7', 'request_id': 'req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:24.668621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] The token cache is being invalidate because project 2e2602783b8a4c20a292f9912457a2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:24.668904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:24.669552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6439c66a-512b-4e6a-b26f-c3b0ecfac2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:24.671061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1002/2004] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:24 2022] DELETE /identity/v3/projects/2e2602783b8a4c20a292f9912457a2d7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:24.675989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:24.676271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:24.676567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:24.694914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:24.696101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:24.753797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1003/2005] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:24.761379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:24.761674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:24.761950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:24.782018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:24.783274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:24.791207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1003/2006] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:24.795317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:24.806017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10cec6-66b7-445b-bd02-1e7f3643d181 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:24.810552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:24.811200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:24.811482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:24.811754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:24.813165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:24.813476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:24.815090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:24.827100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1004/2007] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:50:24.831890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:24.841326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b70dd-f51d-41ac-afb2-54e212b32eb6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:24.846311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:24.846966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:24.847249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:24.847518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:24.848251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:24.850243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:24.854442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:24.946025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1004/2008] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:24 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:24.950215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:24.961359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3551c2-0670-40ca-927d-d7bf3b9da0eb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:25.017544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:25.018245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.018537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.018813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.019518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:25.021523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:25.120750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1005/2009] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:24 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:25.125801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:25.136066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793156-3bab-40dc-9f48-de2d4ab12535 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:25.141283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:25.141947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.142231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.142560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.143271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:25.144903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:25.154971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1005/2010] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:25.158902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:25.168825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b246b62-1d44-4960-a1bb-3e14b1e0e9eb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:25.174978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:25.175751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.176135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.176412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.177200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:25.179090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb09c8-35ab-4a66-9c94-4a0807878244 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:25.190703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1006/2011] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:25.194718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:25.204841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73830f2-1995-41f6-a512-9e5a0b684eee admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:25.209455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:25.210347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.210628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.210897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] PATH_INFO: `/v3/projects/6b5812fce5574d358fa96c7269da086a/users/a66fdd9c629842fb89882c9425026b71/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.227048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5812fce5574d358fa96c7269da086a, user_id=a66fdd9c629842fb89882c9425026b71, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:25.231196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:25.293086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1006/2012] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:25 2022] PUT /identity/v3/projects/6b5812fce5574d358fa96c7269da086a/users/a66fdd9c629842fb89882c9425026b71/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:25.304725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.305017 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.305318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.333484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] MFA Rules not processed for user `a66fdd9c629842fb89882c9425026b71`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:25.334810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:25.347898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:25.400042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1007/2013] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:25.412731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:25.414562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:25.473300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2c48-48ae-4ca4-85e4-e6901b85aec0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:25.477099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:25.477756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:25.478042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:25.478319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:25.488358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:25.489849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:25.499049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1007/2014] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:35.158341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:35.162828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:35.245325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc8566-56dc-4198-be62-5ac51c4b29dd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:35.250010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:35.251000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:35.251288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:35.251568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] PATH_INFO: `/v3/users/f1c789055f344bb59a86b6d4fe5cac91` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:35.253004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c789055f344bb59a86b6d4fe5cac91)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:35.254988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:35.390908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c789055f344bb59a86b6d4fe5cac91'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.402508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c789055f344bb59a86b6d4fe5cac91'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.403196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c789055f344bb59a86b6d4fe5cac91'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.403803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c789055f344bb59a86b6d4fe5cac91'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.411334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d544f2f5-70f1-4ccc-9ba9-bf5a78cfdb9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c789055f344bb59a86b6d4fe5cac91'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.474458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1008/2015] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:35 2022] DELETE /identity/v3/users/f1c789055f344bb59a86b6d4fe5cac91 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:35.494845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:35.716257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:35.729170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bf6-e50d-4b24-83b8-11f0fb7bafa4 service glance] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:35.787389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:35.788252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:35.788534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:35.788812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] PATH_INFO: `/v3/projects/5a6b49c60dc845c5a74f3ab5d132740d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:35.790012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6b49c60dc845c5a74f3ab5d132740d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:35.791945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:35.910905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6b49c60dc845c5a74f3ab5d132740d', 'request_id': 'req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:35.917074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6b49c60dc845c5a74f3ab5d132740d', 'request_id': 'req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:36.000230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6b49c60dc845c5a74f3ab5d132740d', 'request_id': 'req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:36.006125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] The token cache is being invalidate because project 5a6b49c60dc845c5a74f3ab5d132740d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:36.006412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:36.007043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fb17d39b-7ae0-4a4d-a3fd-6dc22e3b3ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:36.008591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1008/2016] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:35 2022] DELETE /identity/v3/projects/5a6b49c60dc845c5a74f3ab5d132740d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:36.013752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.014040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.014364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.032783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:36.034010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:36.066124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:36.100478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1009/2017] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:50:36.104670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.104965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.105323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.125711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:36.127015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:36.135134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1009/2018] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:36.139434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.149861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae8690-b668-4fbb-a176-f9c9e7c7fd2a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.154784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.155435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.155715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.156512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.157264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:36.163241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.163241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.170606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1010/2019] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:50:36.175940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.186830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14079198-b9f4-4bdd-a418-6e8519dd30ec None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.191691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.192342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.192626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.192902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.193628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.195556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.299013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1010/2020] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:36.303081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.313681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9309434-a119-4019-84b2-a34e4cf3190a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.370160 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.370831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.371164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.371443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.372147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.374128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.461019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1011/2021] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:50:36.462453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.473870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c405696-1ce4-472b-924f-f8980632b495 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.478678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.479346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.479632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.479913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.480644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.482288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.492990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1011/2022] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:36.497076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.507642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e0f9-5841-446f-854d-26d34d1a5667 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.512600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.513200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.513485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.513764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.514490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.516591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2eb3e-2203-4063-940c-efcbc93d88eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.527473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1012/2023] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:36.531897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.541939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc4a7-24da-4fee-b76b-dccd1357baaf admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.546545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.547475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.547754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.548027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] PATH_INFO: `/v3/projects/10b3184b00df4a8c96e2056ec65551fc/users/12a4d4e740554bfc973efd08633378fa/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.563706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b3184b00df4a8c96e2056ec65551fc, user_id=12a4d4e740554bfc973efd08633378fa, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.565425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.685935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1012/2024] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:36 2022] PUT /identity/v3/projects/10b3184b00df4a8c96e2056ec65551fc/users/12a4d4e740554bfc973efd08633378fa/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:36.696817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.697143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.697429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.724396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] MFA Rules not processed for user `12a4d4e740554bfc973efd08633378fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:36.725642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:36.791025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1013/2025] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:36.795387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.824530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:36.855193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d724a-ef2a-44af-acbb-252293010431 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.860459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:36.861129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:36.861412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:36.861686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:36.862366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:36.864245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:36.939350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1013/2026] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:36.943264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:36.954429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dc692-d296-4771-bd36-9ddf6c3f7694 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:36.982692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:37.009568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.010249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.010579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.010859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.011556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.013509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.081451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1014/2027] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:36 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:37.085224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.096007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbceaa-9db8-4b48-9573-c2faacdf9c17 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.100596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.101286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.101598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.101871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.102587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.104157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.115932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1014/2028] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:37.119930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.130513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd5ca-1321-4dcc-95e1-4b12ce5ad730 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.135066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.135929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.136207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.136478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] PATH_INFO: `/v3/projects/3da5dffb2d914f2c941d13322d40ee0f/users/da3a5571f06040e2ba707ca853749d49/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.151206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da5dffb2d914f2c941d13322d40ee0f, user_id=da3a5571f06040e2ba707ca853749d49, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.153892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.228851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1015/2029] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:37 2022] PUT /identity/v3/projects/3da5dffb2d914f2c941d13322d40ee0f/users/da3a5571f06040e2ba707ca853749d49/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:37.232796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.243828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b88f-368c-4ddf-b188-a27ba2ff77c6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.299634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.300288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.300567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.300837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.301585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.303159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f6c5-a5a7-451c-8886-c18f4558583c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.313666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1015/2030] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:37.317676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.327934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd2bdd-3ea1-43ae-959e-eb526e3308bc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.332615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.333489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.333768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.334040 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] PATH_INFO: `/v3/projects/3da5dffb2d914f2c941d13322d40ee0f/users/da3a5571f06040e2ba707ca853749d49/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.336010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da5dffb2d914f2c941d13322d40ee0f, user_id=da3a5571f06040e2ba707ca853749d49, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.338672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.349239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1016/2031] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:37 2022] PUT /identity/v3/projects/3da5dffb2d914f2c941d13322d40ee0f/users/da3a5571f06040e2ba707ca853749d49/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:37.360966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.361289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.361666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.388693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] MFA Rules not processed for user `da3a5571f06040e2ba707ca853749d49`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:37.389930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:37.458938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1016/2032] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:37.470796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.472477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:37.529485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8499c-8566-4af5-80a0-b1694e35d78a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.534009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.534654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.534933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.535206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.544857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.546296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c2d81-a3ab-42e8-802b-11e0cc93c3fd service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.557558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1017/2033] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:37.622529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:37.648655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:37.687129 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff922cbb-1ba1-4fc7-8630-70fab7a03a7f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:37.692158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:37.693094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.693456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.693737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] PATH_INFO: `/v3/users/12a4d4e740554bfc973efd08633378fa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.694984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a4d4e740554bfc973efd08633378fa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:37.696896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:37.843735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a4d4e740554bfc973efd08633378fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:37.849664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a4d4e740554bfc973efd08633378fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:37.862031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a4d4e740554bfc973efd08633378fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:37.862710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a4d4e740554bfc973efd08633378fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:37.948433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a4d4e740554bfc973efd08633378fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:37.955810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1017/2034] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:37 2022] DELETE /identity/v3/users/12a4d4e740554bfc973efd08633378fa => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:37.960347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:37.964235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:37.964235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:37.979171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:37.980372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:38.037576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1018/2035] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:38.047029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:38.048637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:38.112626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035decd-02c0-45bd-b0b1-8fadec6ac44a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:38.115412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:38.116063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:38.116345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:38.116621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:38.126266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:38.127680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:38.137618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1018/2036] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:38.532949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:38.537227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:38.546699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a6ef-bbe9-4671-9c58-ee4798cfeb6e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:38.552056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:38.552928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:38.553735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:38.553851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] PATH_INFO: `/v3/projects/10b3184b00df4a8c96e2056ec65551fc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:38.555051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b3184b00df4a8c96e2056ec65551fc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:38.556977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:38.581910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:38.651888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b3184b00df4a8c96e2056ec65551fc', 'request_id': 'req-16a558bc-e244-4f40-aca5-8dd0c3d67a33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.658236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b3184b00df4a8c96e2056ec65551fc', 'request_id': 'req-16a558bc-e244-4f40-aca5-8dd0c3d67a33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.697047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b3184b00df4a8c96e2056ec65551fc', 'request_id': 'req-16a558bc-e244-4f40-aca5-8dd0c3d67a33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.708827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] The token cache is being invalidate because project 10b3184b00df4a8c96e2056ec65551fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:38.709153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.709808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.711370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1019/2037] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:38 2022] DELETE /identity/v3/projects/10b3184b00df4a8c96e2056ec65551fc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:38.715378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:38.725222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b96d84-e931-4700-9cca-9d6a4c47dcba service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:38.768430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:38.784261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:38.785193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:38.785476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:38.785748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] PATH_INFO: `/v3/users/da3a5571f06040e2ba707ca853749d49` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:38.786951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3a5571f06040e2ba707ca853749d49)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:38.789658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:38.959536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3a5571f06040e2ba707ca853749d49'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.966494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3a5571f06040e2ba707ca853749d49'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.979104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3a5571f06040e2ba707ca853749d49'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:38.979768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3a5571f06040e2ba707ca853749d49'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.045256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f30429e-f1da-47ac-97ea-36ba68abfce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3a5571f06040e2ba707ca853749d49'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.052248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1019/2038] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:38 2022] DELETE /identity/v3/users/da3a5571f06040e2ba707ca853749d49 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:39.435925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:39.437395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:39.450408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a558bc-e244-4f40-aca5-8dd0c3d67a33 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:39.517327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:39.518193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:39.518523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:39.518797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] PATH_INFO: `/v3/projects/3da5dffb2d914f2c941d13322d40ee0f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:39.520009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da5dffb2d914f2c941d13322d40ee0f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:39.522004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:39.618313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da5dffb2d914f2c941d13322d40ee0f', 'request_id': 'req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.624463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da5dffb2d914f2c941d13322d40ee0f', 'request_id': 'req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.691852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da5dffb2d914f2c941d13322d40ee0f', 'request_id': 'req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.702969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] The token cache is being invalidate because project 3da5dffb2d914f2c941d13322d40ee0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:39.703871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.704595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0802b1b3-4acb-451b-bf33-a1a79d6c31a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:39.706259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1020/2039] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:39 2022] DELETE /identity/v3/projects/3da5dffb2d914f2c941d13322d40ee0f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:39.711127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:39.711531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:39.711814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:39.729997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:39.731269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:39.793613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1020/2040] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:39.801224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:39.801511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:39.801782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:39.820646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:39.821880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:39.830363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1021/2041] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:39.834940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:39.844926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d5de9-cd4a-4ee0-839e-146273759c7d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:39.849534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:39.850185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:39.850468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:39.850749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:39.851468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:50:39.851774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:39.853400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:39.866499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1021/2042] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:50:39.870879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:39.881429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ac45-7e65-4860-b8d6-41913ed85501 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:39.886683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:39.887369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:39.887655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:39.888044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:39.888799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:39.892139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:39.997097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1022/2043] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:39 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:40.001549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.015403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b524f1-7633-48b2-a810-007fe7455733 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.064075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:40.073853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.074624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.074913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.075226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.075921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.077958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.183151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1022/2044] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:40 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:40.184686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.196157 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e529-2188-4b8b-9153-64c7b7141fff admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.200987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.205824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.205824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.205824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.205824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.205824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.213107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:40.219930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1023/2045] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:50:40.223400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.233680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a3c2d-afdb-4dfd-a49a-f0db05d7098b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.238995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.239655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.239941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.240277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.241002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.242626 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c249975-a2f5-4e68-b02b-1390cd042c3e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.257114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1023/2046] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:40.260984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.273201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a22be-9882-432e-954b-36c19e9b28ef admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.277990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.278933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.279213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.279485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] PATH_INFO: `/v3/projects/6ffac659182043549b1b545ab66bfb3e/users/8195b08f02e94f4ca52ed1b56152f73f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.294944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffac659182043549b1b545ab66bfb3e, user_id=8195b08f02e94f4ca52ed1b56152f73f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.297748 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.360384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1024/2047] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:40 2022] PUT /identity/v3/projects/6ffac659182043549b1b545ab66bfb3e/users/8195b08f02e94f4ca52ed1b56152f73f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:40.370885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f650a-267f-452c-973e-095970b93eea None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.371176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f650a-267f-452c-973e-095970b93eea None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.371456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f650a-267f-452c-973e-095970b93eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.397759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dd0f650a-267f-452c-973e-095970b93eea None None] MFA Rules not processed for user `8195b08f02e94f4ca52ed1b56152f73f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:40.398961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dd0f650a-267f-452c-973e-095970b93eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:40.465058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1024/2048] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:40.469998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.532084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eaf81-9cd4-4491-a654-186a4a6e12cb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.536635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.537306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.537613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.537886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.538571 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.540462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.620317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1025/2049] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:50:40 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:40.623442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f650a-267f-452c-973e-095970b93eea None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.635056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f650a-267f-452c-973e-095970b93eea None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.692891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.693586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.693872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.694148 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.694847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.696770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.782815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1025/2050] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:50:40 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:50:40.786503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.797769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b36a-616c-4de6-a104-1b344d7d5937 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.802639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.803564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.803873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.804144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.804895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.806518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.818086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1026/2051] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:40.823104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.833477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae34-0df5-46ad-a207-03866a2d4be0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:40.838099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:40.838984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:40.839267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:40.839544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] PATH_INFO: `/v3/projects/e2fae0e21d334e39904f71c5507c825f/users/465b3eaf508b4340b3480603d4ff8359/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:40.853923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fae0e21d334e39904f71c5507c825f, user_id=465b3eaf508b4340b3480603d4ff8359, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:40.856572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:40.961314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1026/2052] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:40 2022] PUT /identity/v3/projects/e2fae0e21d334e39904f71c5507c825f/users/465b3eaf508b4340b3480603d4ff8359/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:40.965597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:40.976474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793a8b-c10f-48bd-a039-cdef833bb0a0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:41.034246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:41.034825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:41.035106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:41.035375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:41.036094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:41.037705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8d937-3576-4380-b9d5-d7d8ea02e0e2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:41.048162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1027/2053] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:50:41.052271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:41.065689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b833ee-e837-49b2-bc98-b762112463c7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:41.071356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:41.072238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:41.072519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:41.072794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] PATH_INFO: `/v3/projects/e2fae0e21d334e39904f71c5507c825f/users/465b3eaf508b4340b3480603d4ff8359/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:41.074807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fae0e21d334e39904f71c5507c825f, user_id=465b3eaf508b4340b3480603d4ff8359, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:41.077517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:41.089461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1027/2054] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:50:41 2022] PUT /identity/v3/projects/e2fae0e21d334e39904f71c5507c825f/users/465b3eaf508b4340b3480603d4ff8359/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:50:41.100897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:41.101223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:41.101504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:41.131580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] MFA Rules not processed for user `465b3eaf508b4340b3480603d4ff8359`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:41.132864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:41.168915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:41.208919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1028/2055] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:41.222195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:41.224250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:41.243912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:41.290907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90d607-1559-465e-bf12-fe5c866524c8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:41.294530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:41.295172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:41.295452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:41.295728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:41.305843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:41.307348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:41.318049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1028/2056] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:41.418463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:41.428886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb853d-4567-4afb-84b9-ecea41637815 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:41.432740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:41.433641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:41.433930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:41.434491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:41.493805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:41.495255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74987ae8-7acb-42df-9eed-deade67b5f8e service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:41.504739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1029/2057] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:51.313650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=21 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:51.500405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=41 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:58.131104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:58.193719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ab480-f6d4-4a71-93bf-985c059faa37 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:58.198629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:58.199495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:58.199778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:58.200051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] PATH_INFO: `/v3/users/8195b08f02e94f4ca52ed1b56152f73f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:58.201270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8195b08f02e94f4ca52ed1b56152f73f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:58.203698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:58.390302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8195b08f02e94f4ca52ed1b56152f73f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:58.397050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8195b08f02e94f4ca52ed1b56152f73f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:58.409209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8195b08f02e94f4ca52ed1b56152f73f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:58.409880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8195b08f02e94f4ca52ed1b56152f73f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:58.478302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8195b08f02e94f4ca52ed1b56152f73f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:58.487237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1029/2058] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:58 2022] DELETE /identity/v3/users/8195b08f02e94f4ca52ed1b56152f73f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:58.490561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:58.491026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:58.491118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:58.508922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:50:58.510153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:58.573804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1030/2059] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:50:58.586602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:58.588063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:58.671835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130198-a946-45d4-8723-3752f07637ff admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:58.681724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:58.682452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:58.683308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:58.683308 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:58.703236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:58.704878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:58.715598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1030/2060] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:50:59.207561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:59.209684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:50:59.222851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9b1e-cf45-4c51-a836-3680a7ef1d22 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:59.228745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:59.229320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:59.229602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:59.230025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] PATH_INFO: `/v3/projects/6ffac659182043549b1b545ab66bfb3e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:59.231173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffac659182043549b1b545ab66bfb3e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:59.233982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:59.360584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffac659182043549b1b545ab66bfb3e', 'request_id': 'req-25f47564-22c1-4277-b20f-7a9b6b3e6278'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.366663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffac659182043549b1b545ab66bfb3e', 'request_id': 'req-25f47564-22c1-4277-b20f-7a9b6b3e6278'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.468802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffac659182043549b1b545ab66bfb3e', 'request_id': 'req-25f47564-22c1-4277-b20f-7a9b6b3e6278'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.480825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] The token cache is being invalidate because project 6ffac659182043549b1b545ab66bfb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:50:59.481199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.481845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.483956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1031/2061] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:50:59 2022] DELETE /identity/v3/projects/6ffac659182043549b1b545ab66bfb3e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:50:59.487980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:50:59.497757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0475d-200c-45cc-be4f-900202f0d5a8 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:50:59.516055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:50:59.554697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:50:59.555565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:50:59.555848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:50:59.556123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] PATH_INFO: `/v3/users/465b3eaf508b4340b3480603d4ff8359` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:50:59.557336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465b3eaf508b4340b3480603d4ff8359)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:50:59.559266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:50:59.778894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465b3eaf508b4340b3480603d4ff8359'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.786546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465b3eaf508b4340b3480603d4ff8359'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.799787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465b3eaf508b4340b3480603d4ff8359'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.800557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465b3eaf508b4340b3480603d4ff8359'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.867825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-96249598-876b-49bb-a6e6-713cf411b574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465b3eaf508b4340b3480603d4ff8359'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:50:59.877270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1031/2062] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:50:59 2022] DELETE /identity/v3/users/465b3eaf508b4340b3480603d4ff8359 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:00.298526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:00.299967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:00.313828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47564-22c1-4277-b20f-7a9b6b3e6278 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:00.372481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:00.373375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.373659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.375578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] PATH_INFO: `/v3/projects/e2fae0e21d334e39904f71c5507c825f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.376785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2fae0e21d334e39904f71c5507c825f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:00.378717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:00.470772 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:00.487632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fae0e21d334e39904f71c5507c825f', 'request_id': 'req-801de86d-a80f-4cec-b83f-c861f32d7f00'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:00.494287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fae0e21d334e39904f71c5507c825f', 'request_id': 'req-801de86d-a80f-4cec-b83f-c861f32d7f00'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:00.560196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fae0e21d334e39904f71c5507c825f', 'request_id': 'req-801de86d-a80f-4cec-b83f-c861f32d7f00'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:00.571751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] The token cache is being invalidate because project e2fae0e21d334e39904f71c5507c825f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:00.572033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:00.572690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-801de86d-a80f-4cec-b83f-c861f32d7f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:00.574257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1032/2063] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:00 2022] DELETE /identity/v3/projects/e2fae0e21d334e39904f71c5507c825f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:00.582779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8501-9d13-499f-8206-b105761b371d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.583245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8501-9d13-499f-8206-b105761b371d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.583352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed8501-9d13-499f-8206-b105761b371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.601386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6aed8501-9d13-499f-8206-b105761b371d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:00.605079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6aed8501-9d13-499f-8206-b105761b371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:00.661065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1032/2064] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:00.668877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.669201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.669504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.689882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:00.691075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:00.700464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1033/2065] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:00.705079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8501-9d13-499f-8206-b105761b371d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:00.714698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed8501-9d13-499f-8206-b105761b371d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:00.719843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:00.720498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.720785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.721164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.721891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:00.722198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:00.723817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:00.739767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1033/2066] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:00.743383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:00.757843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce604717-bfa8-4e8f-b673-f4ce388f52e9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:00.762895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:00.763747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.764043 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.764453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.765172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:00.767103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:00.848007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1034/2067] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:00 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:00.852183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:00.862752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3c5cf-07c2-46d3-913d-537bab3edc9f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:00.888645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:00.923393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:00.924094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:00.924431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:00.924716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:00.925491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:00.927715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.022528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1034/2068] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:00 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:01.028973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.040002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf340-1597-4bcd-9e6b-3bd68d20dbfb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.047598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.048536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.048536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.048818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.049560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.051414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.064050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1035/2069] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:01.068136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.078916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef3c73-6aee-473b-bd74-f5d32691c762 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.083581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.084236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.084517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.084790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.085551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.087134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d8708-3850-4b5c-980a-6dbcd5f94afe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.098521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1035/2070] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:01.102607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.114959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877235-52fe-4cf8-923f-31f78b5aab99 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.120116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.121009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.121318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.121592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] PATH_INFO: `/v3/projects/11fcca540c284dc08fa3db99e1174b8d/users/fdec3531f6cc46dc95e9a31d34efb704/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.138507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fcca540c284dc08fa3db99e1174b8d, user_id=fdec3531f6cc46dc95e9a31d34efb704, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.141636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.245152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1036/2071] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:01 2022] PUT /identity/v3/projects/11fcca540c284dc08fa3db99e1174b8d/users/fdec3531f6cc46dc95e9a31d34efb704/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:01.255486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.255771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.256048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.283716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] MFA Rules not processed for user `fdec3531f6cc46dc95e9a31d34efb704`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:01.284938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:01.353708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1036/2072] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:01.358863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.422124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881edb6-a6b0-4ca6-9867-077bf7c1d119 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.427182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.428683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.428978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.429276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.430207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.432289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.532930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1037/2073] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:01 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:01.537121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.548164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44552c-fbb2-4442-87eb-a16996f087f8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.609367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.610018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.610307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.610579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.611277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.613299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.690736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1037/2074] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:01 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:01.694937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.706073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fe25-f32a-452c-a975-2370a5e44133 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.710681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.711338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.711624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.711902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.712695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.714333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.725778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1038/2075] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:01.729950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.740708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c92e3-045f-4183-a2fa-2a1825c5f77c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.745341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.746219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.746513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.746766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] PATH_INFO: `/v3/projects/24c81829be59427a942bd75a4bd61b5e/users/7d4ad063457c4564b711ead364a19ca4/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.764997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c81829be59427a942bd75a4bd61b5e, user_id=7d4ad063457c4564b711ead364a19ca4, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.768270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.814993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.816734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:01.830078 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8b0c6-5846-4c2a-afc9-a6076d3cf370 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:01.834729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:01.835588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:01.835866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:01.836138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] PATH_INFO: `/v3/users/4940de9ddc574406b7ec80075762a7dc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:01.837362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4940de9ddc574406b7ec80075762a7dc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:01.839260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:01.898660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:01.989013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1038/2076] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:01 2022] PUT /identity/v3/projects/24c81829be59427a942bd75a4bd61b5e/users/7d4ad063457c4564b711ead364a19ca4/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:01.992831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:01.996926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:02.066123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c576fb-29b4-4f1a-a207-725e5542290c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:02.071164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:02.071819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.072102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.072370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.073162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:02.074753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:02.085448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1039/2077] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:02.089587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:02.099036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424361db-86f3-4150-a211-81aa0258c33a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:02.103902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:02.104745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.105027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.105333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] PATH_INFO: `/v3/projects/24c81829be59427a942bd75a4bd61b5e/users/7d4ad063457c4564b711ead364a19ca4/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.107349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c81829be59427a942bd75a4bd61b5e, user_id=7d4ad063457c4564b711ead364a19ca4, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:02.110104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2ca1a-0f74-479e-afd1-c01e5c9bcdbe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:02.119975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1040/2078] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:02 2022] PUT /identity/v3/projects/24c81829be59427a942bd75a4bd61b5e/users/7d4ad063457c4564b711ead364a19ca4/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:02.131545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.131835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.132118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.160924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] MFA Rules not processed for user `7d4ad063457c4564b711ead364a19ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:02.162184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:02.184635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4940de9ddc574406b7ec80075762a7dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:02.199858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4940de9ddc574406b7ec80075762a7dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:02.200592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4940de9ddc574406b7ec80075762a7dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:02.201239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4940de9ddc574406b7ec80075762a7dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:02.209667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-850d5ef4-6532-4df5-95e7-8fc90586d246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4940de9ddc574406b7ec80075762a7dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:02.247223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1041/2079] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:02.259971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:02.261436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:02.266135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1039/2080] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:01 2022] DELETE /identity/v3/users/4940de9ddc574406b7ec80075762a7dc => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:02.270196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.270552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.270894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.296129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:02.309189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:02.330536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7f266-a980-4ffe-9a04-258e4a433293 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:02.355699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:02.379502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1040/2081] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:02.386231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:02.387230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.387664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.388775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:02.389380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.390942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:02.460754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d87be-3511-44a8-b836-bc2c731195e9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:02.461093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:02.462667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:02.464717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:02.465463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.465930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.466280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.474719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1042/2082] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:02.481552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:02.483039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:02.493701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1041/2083] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:02.966952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:02.966952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:02.985854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dfc7b-e703-487b-b51a-98d6e44e2357 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:02.990550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:02.991413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:02.991693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:02.991997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] PATH_INFO: `/v3/projects/9195c8ce0ce54fb79917c5c2a8627a7b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:02.994174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9195c8ce0ce54fb79917c5c2a8627a7b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:02.996672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:03.125946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9195c8ce0ce54fb79917c5c2a8627a7b', 'request_id': 'req-074ca4d6-d37f-40fb-ad37-589934c73902'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.132109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9195c8ce0ce54fb79917c5c2a8627a7b', 'request_id': 'req-074ca4d6-d37f-40fb-ad37-589934c73902'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.158548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:03.160282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:03.173367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a04d9-a73e-4c96-82d3-f775bcf11731 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:03.212006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:03.229899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:03.230555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:03.230836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:03.231110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:03.233482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:03.245295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9195c8ce0ce54fb79917c5c2a8627a7b', 'request_id': 'req-074ca4d6-d37f-40fb-ad37-589934c73902'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.259568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] The token cache is being invalidate because project 9195c8ce0ce54fb79917c5c2a8627a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:03.259960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.261130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.263334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1043/2084] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:02 2022] DELETE /identity/v3/projects/9195c8ce0ce54fb79917c5c2a8627a7b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:03.268538 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:03.269979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:03.310667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:03.314363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:03.323721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1042/2085] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:03.347648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ca4d6-d37f-40fb-ad37-589934c73902 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:03.352377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:03.353287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:03.353577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:03.353850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] PATH_INFO: `/v3/users/849c9c6058b14fe8a32fedb6c053bf97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:03.355045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849c9c6058b14fe8a32fedb6c053bf97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:03.356948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:03.359098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:03.597262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849c9c6058b14fe8a32fedb6c053bf97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.604551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849c9c6058b14fe8a32fedb6c053bf97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.617481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849c9c6058b14fe8a32fedb6c053bf97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.619911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849c9c6058b14fe8a32fedb6c053bf97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.725126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a604043c-bb04-46b7-8786-28f19d3701e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849c9c6058b14fe8a32fedb6c053bf97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:03.731422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1044/2086] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:03 2022] DELETE /identity/v3/users/849c9c6058b14fe8a32fedb6c053bf97 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:04.184228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:04.196219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36e68f-0e59-4816-a2ce-326c01347240 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:04.261081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:04.261978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.262255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.262526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] PATH_INFO: `/v3/projects/38e995173ed9423195e0d129f8454b64` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.263700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e995173ed9423195e0d129f8454b64)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:04.266619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:04.382104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e995173ed9423195e0d129f8454b64', 'request_id': 'req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:04.388125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e995173ed9423195e0d129f8454b64', 'request_id': 'req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:04.473858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e995173ed9423195e0d129f8454b64', 'request_id': 'req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:04.485802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] The token cache is being invalidate because project 38e995173ed9423195e0d129f8454b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:04.486089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:04.486720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0c3c6059-a5d8-425c-9a60-dfb6b9b860f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:04.488245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1043/2087] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:04 2022] DELETE /identity/v3/projects/38e995173ed9423195e0d129f8454b64 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:04.493083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.493576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.493674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.512658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:04.513989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:04.577117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1045/2088] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:04.586664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.586664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.586906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.607088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:04.607088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:04.616792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1044/2089] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:04.621250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:04.631966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5c59-9383-4d91-9e1e-cd9051dcaea0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:04.637265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:04.638020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.641450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.641888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.642715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:04.643138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:04.644793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:04.660934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1046/2090] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:51:04.665799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:04.677984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e70755-609a-4bd2-96f2-7966e8ff2f0d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:04.702664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:04.703347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.703684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.703915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.704627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:04.706677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:04.722010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:04.812946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1045/2091] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:04 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:04.816861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:04.826984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340aa56-d2c4-49e0-8f0f-12460ed308d4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:04.885718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:04.886413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.886712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.886997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.887709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:04.889842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:04.900052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:04.975688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1047/2092] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:04 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:04.979906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:04.990968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20197-908b-49b0-9fb3-2e01fa01f8e0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:04.996071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:04.996071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:04.996444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:04.996617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:04.997367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.003454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.010322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1046/2093] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:05.014036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.024772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c603e4-77a9-48e6-bf34-9198e27fa7ad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.029565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.030737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.031018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.031292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.032011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.033624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cbf7a-a7ac-49aa-a528-bbc866d1bb25 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.044353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1048/2094] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:05.049588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.064173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c27b9-9d35-42a7-8bcb-2db2749523fc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.069262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.070967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.070967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.070967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] PATH_INFO: `/v3/projects/4e40f7b8fdbb4eaf81276f6da52d4b4a/users/27ebb79437f243aea8c9c74679873d8c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.093198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e40f7b8fdbb4eaf81276f6da52d4b4a, user_id=27ebb79437f243aea8c9c74679873d8c, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.096026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.188551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1047/2095] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:05 2022] PUT /identity/v3/projects/4e40f7b8fdbb4eaf81276f6da52d4b4a/users/27ebb79437f243aea8c9c74679873d8c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:05.198555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.198848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.199128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.227606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] MFA Rules not processed for user `27ebb79437f243aea8c9c74679873d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:05.228815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:05.295191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1049/2096] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:05.299911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.363194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cd86-e999-4522-9739-834eb9f15453 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.367906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.369508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.369508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.369508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.369849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.371769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.470076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1048/2097] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:05 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:05.474406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.485408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b680f8-04ec-4085-9595-f371566ee9df None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.544840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.545677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.545903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.546177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.546875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.548784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.674421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1050/2098] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:05 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:05.688127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.700467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5b86a-51d0-4ae6-ab69-92ea57264dfb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.712020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.712020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.712020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.712020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.712370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.712370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.728717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1049/2099] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:05.732983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.743164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b76ec-e68b-433b-ba48-ecdfc44709fd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.747794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.749388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.749787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.750062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] PATH_INFO: `/v3/projects/5b3c366721304c4a894f143c3cedc09d/users/7d51291dad504fe494611d28d5831ffc/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.765798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3c366721304c4a894f143c3cedc09d, user_id=7d51291dad504fe494611d28d5831ffc, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.768427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.834110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1051/2100] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:05 2022] PUT /identity/v3/projects/5b3c366721304c4a894f143c3cedc09d/users/7d51291dad504fe494611d28d5831ffc/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:05.838027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.848058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7868be5-38bd-4a82-b491-03903499ece7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.905264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.905963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.906250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.906528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.907250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.908873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978379-725c-4bf6-9df5-4974b0e42c77 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.918749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1050/2101] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:05.923460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:05.934014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbfab6-3f5b-4bf1-b38b-f87d999a6e33 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:05.939315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:05.940191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.940472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.940744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] PATH_INFO: `/v3/projects/5b3c366721304c4a894f143c3cedc09d/users/7d51291dad504fe494611d28d5831ffc/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.942759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3c366721304c4a894f143c3cedc09d, user_id=7d51291dad504fe494611d28d5831ffc, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:05.945425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:05.956525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1052/2102] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:05 2022] PUT /identity/v3/projects/5b3c366721304c4a894f143c3cedc09d/users/7d51291dad504fe494611d28d5831ffc/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:05.967260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:05.967575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:05.967798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:05.994221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] MFA Rules not processed for user `7d51291dad504fe494611d28d5831ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:05.995425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:06.013741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:06.064373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1051/2103] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:06.075200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:06.076947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:06.108557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:06.137461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e7a0-c051-48c3-b6bb-25d01a6d9242 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:06.141289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:06.144057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:06.144057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:06.144057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:06.152288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:06.153751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2323d-5f21-49ca-b995-1ec6110de6d3 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:06.163943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1053/2104] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:06.333832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:06.399346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be6c14-dd59-443a-8415-577a16e1a75e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:06.403848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:06.405269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:06.405638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:06.405917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] PATH_INFO: `/v3/users/27ebb79437f243aea8c9c74679873d8c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:06.407111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ebb79437f243aea8c9c74679873d8c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:06.409057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:06.578592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ebb79437f243aea8c9c74679873d8c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:06.590687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ebb79437f243aea8c9c74679873d8c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:06.591364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ebb79437f243aea8c9c74679873d8c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:06.592007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ebb79437f243aea8c9c74679873d8c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:06.599727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ebb79437f243aea8c9c74679873d8c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:06.685780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1052/2105] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:06 2022] DELETE /identity/v3/users/27ebb79437f243aea8c9c74679873d8c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:06.690367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:06.690863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:06.691057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:06.716810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:06.716810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:06.781255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1054/2106] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:06.790919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:06.792482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:06.833615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:06.851045 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297032a-4771-49ad-9c7e-7ee5ae405aef admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:06.854841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:06.855518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:06.855801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:06.856074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:06.865366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:06.867421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:06.877423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1053/2107] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:07.291772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:07.293641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:07.312848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e565c-63eb-4216-8cef-32a907dd1d38 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:07.318552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:07.327367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:07.327367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:07.327367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] PATH_INFO: `/v3/projects/4e40f7b8fdbb4eaf81276f6da52d4b4a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:07.327367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e40f7b8fdbb4eaf81276f6da52d4b4a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:07.327367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:07.457904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e40f7b8fdbb4eaf81276f6da52d4b4a', 'request_id': 'req-3c918e06-d6cc-4695-a1e0-52fb508b888d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.464214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e40f7b8fdbb4eaf81276f6da52d4b4a', 'request_id': 'req-3c918e06-d6cc-4695-a1e0-52fb508b888d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.547857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e40f7b8fdbb4eaf81276f6da52d4b4a', 'request_id': 'req-3c918e06-d6cc-4695-a1e0-52fb508b888d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.559315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] The token cache is being invalidate because project 4e40f7b8fdbb4eaf81276f6da52d4b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:07.560283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.560283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.562493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1055/2108] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:07 2022] DELETE /identity/v3/projects/4e40f7b8fdbb4eaf81276f6da52d4b4a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:07.566529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:07.576637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f8da7-dd26-4613-aaee-e3a2a2ee4723 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:07.636708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:07.637656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:07.637942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:07.638246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] PATH_INFO: `/v3/users/7d51291dad504fe494611d28d5831ffc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:07.639577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d51291dad504fe494611d28d5831ffc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:07.641549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:07.804555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d51291dad504fe494611d28d5831ffc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.822257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d51291dad504fe494611d28d5831ffc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.823013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d51291dad504fe494611d28d5831ffc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.823565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d51291dad504fe494611d28d5831ffc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.831343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9a086b96-afd1-4e59-a304-15dc046ba6d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d51291dad504fe494611d28d5831ffc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:07.896251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1054/2109] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:07 2022] DELETE /identity/v3/users/7d51291dad504fe494611d28d5831ffc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:08.658428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:08.660022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:08.674597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c918e06-d6cc-4695-a1e0-52fb508b888d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:08.743577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:08.753157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:08.754171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:08.754462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:08.754742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] PATH_INFO: `/v3/projects/5b3c366721304c4a894f143c3cedc09d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:08.755944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b3c366721304c4a894f143c3cedc09d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:08.757922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:08.896448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b3c366721304c4a894f143c3cedc09d', 'request_id': 'req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:08.913593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b3c366721304c4a894f143c3cedc09d', 'request_id': 'req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:09.000190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b3c366721304c4a894f143c3cedc09d', 'request_id': 'req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:09.011581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] The token cache is being invalidate because project 5b3c366721304c4a894f143c3cedc09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:09.016505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:09.016505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c47c8797-2dd2-4fa1-a472-3fb7ee6478af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:09.016505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1056/2110] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:08 2022] DELETE /identity/v3/projects/5b3c366721304c4a894f143c3cedc09d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:09.020348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.020832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.020935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.038931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:09.040177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:09.097302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:09.106555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1055/2111] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:09.116010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.116362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.116714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.136461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:09.137808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:09.148395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1057/2112] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:09.153134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.167066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d722f9-8ceb-4eea-abd7-5d553233d016 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.172008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.172799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.173121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.173404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.174193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:09.174503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.180675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.205738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1056/2113] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:09.210429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.220749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02b2d8-7c1f-4653-acc8-6e4f6f947434 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.225553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.226211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.226499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.226777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.227472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.229424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.342856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1058/2114] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:09 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:09.347396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.357098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65712e89-54a1-4413-b7ac-47f269b0902e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.413766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.414441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.414798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.415071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.415896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.417789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.531409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1057/2115] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:09 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:09.535926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.550015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c9571-91a1-4e1d-ae28-a912d307886b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.554871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.555522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.555802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.556076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.556793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.558569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.569768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1059/2116] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:09.577417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.588243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0183b-4e56-4d2f-b4b9-a02f83b4b023 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.593197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.593814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.594095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.594500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.595225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.596834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae84476-316f-41b1-a7d8-91647af25f66 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.607095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1058/2117] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:09.611325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.621353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c9007-511f-4291-8155-3a0e3debf0f5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.627356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.628233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.628519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.630190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] PATH_INFO: `/v3/projects/a6a4088a03664df295e7e6a4aac6f79e/users/4f4d80e277a14e58bc31da5bd3fbed28/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.645955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a4088a03664df295e7e6a4aac6f79e, user_id=4f4d80e277a14e58bc31da5bd3fbed28, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.648917 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.753157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1060/2118] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:09 2022] PUT /identity/v3/projects/a6a4088a03664df295e7e6a4aac6f79e/users/4f4d80e277a14e58bc31da5bd3fbed28/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:09.758372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.762171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.762171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.776547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:09.777838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:09.844997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1059/2119] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:09.854682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:09.856404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:09.930988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e8144-3509-41c5-a0a3-5b1d8346e5ea admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:09.934901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:09.935602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:09.936815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:09.936965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:09.947020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:09.948510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12dc72-289c-42e0-97bb-cf01c3fd04c5 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:09.958901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1061/2120] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:10.866520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:10.868393 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:10.885556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d25fc-7233-496c-8e40-b5a7fcf77535 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:10.893717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:10.895289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:10.895633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:10.895992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] PATH_INFO: `/v3/users/fdec3531f6cc46dc95e9a31d34efb704` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:10.897637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdec3531f6cc46dc95e9a31d34efb704)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:10.899579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.133807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdec3531f6cc46dc95e9a31d34efb704'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:11.142049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:11.146573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdec3531f6cc46dc95e9a31d34efb704'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:11.147278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdec3531f6cc46dc95e9a31d34efb704'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:11.147882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdec3531f6cc46dc95e9a31d34efb704'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:11.155232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfb5e8b5-f255-4eba-abdd-d2396ebd6d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdec3531f6cc46dc95e9a31d34efb704'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:11.247988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.247988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.248631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.262922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1060/2121] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:10 2022] DELETE /identity/v3/users/fdec3531f6cc46dc95e9a31d34efb704 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:11.270563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.270855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.271137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.278382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] MFA Rules not processed for user `4f4d80e277a14e58bc31da5bd3fbed28`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:11.280333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:11.290365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:11.291534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:11.291815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:11.353651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1062/2122] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:11.358648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.365111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:11.365261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1061/2123] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:11.374872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.375647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d87d8-6027-453c-a6b2-d931a5360443 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.376279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:11.380244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.381252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.382250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.382415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.383174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.385127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.436638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e736a3c-1797-43ee-8427-0e3993d5241f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.440354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.440994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.441307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.441583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.451026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.452636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.461897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1062/2124] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:11.487518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1063/2125] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:11 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:11.490794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.500868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa172489-f798-49a6-9473-b4179f59b12a service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.562201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.562908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.563188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.563467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.564165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.566165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.639892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1063/2126] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:11 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:11.644411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.656047 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0603e59-4fa8-4d91-aff2-3871d7ac851e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.661010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.664145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.664145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.664145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.664145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.665070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.676584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1064/2127] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:11.680783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.690952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48467c-2bad-4271-a26e-9c8b11e05b35 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.695732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.696611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.696894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.697256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] PATH_INFO: `/v3/projects/5462e375e4b14aeba6059b461442d4d8/users/3427fe2a85f944e597a1e58ff4849bb2/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.701066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:11.712938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5462e375e4b14aeba6059b461442d4d8, user_id=3427fe2a85f944e597a1e58ff4849bb2, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.715760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.798663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1064/2128] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:11 2022] PUT /identity/v3/projects/5462e375e4b14aeba6059b461442d4d8/users/3427fe2a85f944e597a1e58ff4849bb2/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:11.802855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.812842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1884-a8f8-44b0-b1bd-e75ee4f16f9d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.871377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.872260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.872602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.873061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.873920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.875311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.876152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.883334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:11.892596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1065/2129] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:11.898277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:11.909798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f8f1e-0e4e-445b-ad7f-764c6796e600 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.910604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273218f7-ef47-451f-8959-17da6f62b857 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:11.915453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.916214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:11.916368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.917081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.917081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] PATH_INFO: `/v3/projects/11fcca540c284dc08fa3db99e1174b8d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.917292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:11.917553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:11.917833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] PATH_INFO: `/v3/projects/5462e375e4b14aeba6059b461442d4d8/users/3427fe2a85f944e597a1e58ff4849bb2/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:11.918414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fcca540c284dc08fa3db99e1174b8d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.920044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5462e375e4b14aeba6059b461442d4d8, user_id=3427fe2a85f944e597a1e58ff4849bb2, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:11.920845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.922786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:11.936417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1066/2130] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:11 2022] PUT /identity/v3/projects/5462e375e4b14aeba6059b461442d4d8/users/3427fe2a85f944e597a1e58ff4849bb2/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:12.051670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fcca540c284dc08fa3db99e1174b8d', 'request_id': 'req-7ecc4134-4414-44a0-b92e-b21978f89159'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.069425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fcca540c284dc08fa3db99e1174b8d', 'request_id': 'req-7ecc4134-4414-44a0-b92e-b21978f89159'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.171743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fcca540c284dc08fa3db99e1174b8d', 'request_id': 'req-7ecc4134-4414-44a0-b92e-b21978f89159'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.187948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] The token cache is being invalidate because project 11fcca540c284dc08fa3db99e1174b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:12.188220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.188897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.195425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1065/2131] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:11 2022] DELETE /identity/v3/projects/11fcca540c284dc08fa3db99e1174b8d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:12.199299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:12.205354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:12.281652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f745b-98f3-4383-a74e-ee72d89cd563 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:12.281652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:12.339876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:12.340757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:12.342693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:12.342693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] PATH_INFO: `/v3/users/7d4ad063457c4564b711ead364a19ca4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:12.345875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4ad063457c4564b711ead364a19ca4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:12.347848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:12.556400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4ad063457c4564b711ead364a19ca4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.564678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4ad063457c4564b711ead364a19ca4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.576756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4ad063457c4564b711ead364a19ca4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.577494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4ad063457c4564b711ead364a19ca4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.695976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-613afee3-668a-4fd3-a985-d677c9a43a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4ad063457c4564b711ead364a19ca4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:12.703572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1067/2132] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:12 2022] DELETE /identity/v3/users/7d4ad063457c4564b711ead364a19ca4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:13.211111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:13.216627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.217765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.218051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.224200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc4134-4414-44a0-b92e-b21978f89159 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:13.259471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] MFA Rules not processed for user `3427fe2a85f944e597a1e58ff4849bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:13.262572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:13.313103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:13.314923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.315336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.316044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] PATH_INFO: `/v3/projects/24c81829be59427a942bd75a4bd61b5e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.317262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c81829be59427a942bd75a4bd61b5e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:13.319475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:13.344121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1068/2133] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:13.356430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:13.357921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:13.371692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:13.415619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c59ca-29de-4625-a52e-9c009be9df22 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:13.419580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:13.421427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.421607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.421892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.423916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:13.443590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c81829be59427a942bd75a4bd61b5e', 'request_id': 'req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:13.453822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c81829be59427a942bd75a4bd61b5e', 'request_id': 'req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:13.492779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:13.494335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcac357-efa8-45cc-85e6-55570026b66e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:13.504196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1069/2134] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:13.547257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c81829be59427a942bd75a4bd61b5e', 'request_id': 'req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:13.554759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:13.560254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] The token cache is being invalidate because project 24c81829be59427a942bd75a4bd61b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:13.561180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:13.562501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c5ee12e-d69d-40f1-b02f-e40f0ea9be51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:13.564448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1066/2135] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:13 2022] DELETE /identity/v3/projects/24c81829be59427a942bd75a4bd61b5e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:13.569716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.570014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.570327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.589300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:13.590632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:13.664437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1070/2136] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:13.672746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.673058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.673351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.692624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:13.693934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:13.702790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1067/2137] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:13.706944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:13.717004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309d277-e181-49cf-9f58-aa4810178277 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:13.722074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:13.722722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.723000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.723272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.723989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:13.724295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:13.725935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:13.741161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1071/2138] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:13.742383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:13.752376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f7a2-4092-4155-b68f-e7bc32c977fd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:13.756948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:13.757622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.757902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.758173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.758863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:13.760759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:13.897456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1068/2139] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:13 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:13.901628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:13.911937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9f7e9-3daf-42b2-8f7b-0263aa6a7702 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:13.968219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:13.968878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:13.969195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:13.969468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:13.970169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:13.972049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.065983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1072/2140] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:13 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:14.070086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.081602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123511f9-b264-44bd-b711-885c1a49cf90 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.086255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.086903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.087186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.087458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.088182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.089848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.104351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1069/2141] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:14.108965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.118809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058044bf-35ae-4675-bc73-5fd5d9255f25 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.124412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.125093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.125382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.125659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.126390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.127975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add84ab-7beb-4a16-b7a1-bb3dc14b3b0c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.138834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1073/2142] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:14.142998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.165172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cc87f-78b0-4cba-80eb-14fe57fe5b05 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.178093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.179154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.179492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.179804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] PATH_INFO: `/v3/projects/b9f709847d264310b220da49fa7e9460/users/948c1687e49d4fbf81aabd49dc34a3f8/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.196777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f709847d264310b220da49fa7e9460, user_id=948c1687e49d4fbf81aabd49dc34a3f8, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.207305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.216433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:14.275295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1070/2143] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:14 2022] PUT /identity/v3/projects/b9f709847d264310b220da49fa7e9460/users/948c1687e49d4fbf81aabd49dc34a3f8/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:14.287968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.288259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.288547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.315682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] MFA Rules not processed for user `948c1687e49d4fbf81aabd49dc34a3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:14.316927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:14.380293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1074/2144] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:14.384698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.450706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346d0b7-791e-4351-acc1-391526341c0b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.455388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.456044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.456329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.456606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.457334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.459274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.545746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1071/2145] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:14 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:14.550216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.562262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f597f29-8eef-4e93-a586-a275fc8e26db None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.620466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.621177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.621574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.621857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.622573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.624668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.702242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1075/2146] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:14 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:14.706159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.717492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f3066-39ab-42b2-b60b-1cfd1df3b7e1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.722171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.722827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.723114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.723394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.724119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.725739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.736372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1072/2147] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:14.740418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.750586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e10cd-a3d1-456a-b9b9-2726227d6ff4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.755265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.756142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.756428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.756707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] PATH_INFO: `/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790/users/75fa6e3dcd434679b67f5d2e0b3a636d/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.771847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef3cd14d18243ef9db7e23dc25fe790, user_id=75fa6e3dcd434679b67f5d2e0b3a636d, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.774532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.839300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1076/2148] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:14 2022] PUT /identity/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790/users/75fa6e3dcd434679b67f5d2e0b3a636d/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:14.843969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.853994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253227a-729d-4b02-a5ca-54ff93e0b50c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.909973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.910631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.910922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.911202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.911924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.913540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0264ac-d10c-4912-ae02-baf98e64c77a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.923171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1073/2149] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:14.927424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:14.938186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c38ba3-2b02-4d28-80cf-9a6d963a40a4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:14.942795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:14.943677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.943963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.944239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] PATH_INFO: `/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790/users/75fa6e3dcd434679b67f5d2e0b3a636d/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.946237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef3cd14d18243ef9db7e23dc25fe790, user_id=75fa6e3dcd434679b67f5d2e0b3a636d, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:14.948863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:14.959775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1077/2150] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:14 2022] PUT /identity/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790/users/75fa6e3dcd434679b67f5d2e0b3a636d/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:14.970197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:14.970518 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:14.970800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:14.977531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:14.996773 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] MFA Rules not processed for user `75fa6e3dcd434679b67f5d2e0b3a636d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:14.998503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:15.072527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1074/2151] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:15.083879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:15.085617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:15.104804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:15.144255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128ef5-2fd1-41aa-b5cf-ca5da2e3e95e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:15.147973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:15.148624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:15.148910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:15.149219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:15.208523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:15.210255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:15.220918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1078/2152] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:15.698187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:15.699693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:15.713310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be92e11-38a4-4cbb-b8d3-2d19dfb7f854 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:15.717256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:15.717916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:15.718202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:15.718477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:15.720272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:15.793492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:15.795020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50c937-8533-4bb7-8847-26f69d7de0d2 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:15.805864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1075/2153] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:25.215413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=41 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:25.801696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=62 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:26.852215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:26.916304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9810d0a-b215-49cd-9cb7-108560783d3b service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:26.921096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:26.922064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:26.922441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:26.922719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] PATH_INFO: `/v3/users/948c1687e49d4fbf81aabd49dc34a3f8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:26.924498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948c1687e49d4fbf81aabd49dc34a3f8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:26.925915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:27.129069 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948c1687e49d4fbf81aabd49dc34a3f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.137664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948c1687e49d4fbf81aabd49dc34a3f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.151112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948c1687e49d4fbf81aabd49dc34a3f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.154698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948c1687e49d4fbf81aabd49dc34a3f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.237888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948c1687e49d4fbf81aabd49dc34a3f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.244347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1079/2154] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:26 2022] DELETE /identity/v3/users/948c1687e49d4fbf81aabd49dc34a3f8 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:27.248835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:27.249295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:27.249491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:27.266920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:27.268514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:27.329024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1076/2155] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:27.338365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:27.339809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:27.400797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e60e0f-cdd9-4b73-aa01-7000a1a75f93 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:27.404551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:27.405224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:27.405506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:27.405778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:27.415317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:27.416728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:27.431015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1080/2156] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:27.829514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:27.831012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:27.844644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbbb94-eee5-43b4-87ad-dff4ce9ad0d2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:27.849714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:27.850749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:27.851127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:27.851479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] PATH_INFO: `/v3/projects/b9f709847d264310b220da49fa7e9460` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:27.852739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f709847d264310b220da49fa7e9460)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:27.854760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:27.954998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f709847d264310b220da49fa7e9460', 'request_id': 'req-574b956c-3ef2-4502-8ffa-dc7d37c793e6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:27.962694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f709847d264310b220da49fa7e9460', 'request_id': 'req-574b956c-3ef2-4502-8ffa-dc7d37c793e6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.049890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f709847d264310b220da49fa7e9460', 'request_id': 'req-574b956c-3ef2-4502-8ffa-dc7d37c793e6'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.064010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] The token cache is being invalidate because project b9f709847d264310b220da49fa7e9460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:28.064305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.064962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.067056 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1077/2157] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:27 2022] DELETE /identity/v3/projects/b9f709847d264310b220da49fa7e9460 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:28.070850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:28.081314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2747ce-2539-482c-9a18-b99faa89d058 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:28.100298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:28.140333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:28.141458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:28.141742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:28.142018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] PATH_INFO: `/v3/users/75fa6e3dcd434679b67f5d2e0b3a636d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:28.143245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fa6e3dcd434679b67f5d2e0b3a636d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:28.145226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:28.360945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fa6e3dcd434679b67f5d2e0b3a636d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.369516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fa6e3dcd434679b67f5d2e0b3a636d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.382745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fa6e3dcd434679b67f5d2e0b3a636d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.386344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fa6e3dcd434679b67f5d2e0b3a636d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.483049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f2089ddc-22b1-4ff6-b48e-6cd66db7b40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fa6e3dcd434679b67f5d2e0b3a636d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:28.491275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1081/2158] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:28 2022] DELETE /identity/v3/users/75fa6e3dcd434679b67f5d2e0b3a636d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:28.965418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:28.967451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:28.982105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b956c-3ef2-4502-8ffa-dc7d37c793e6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.040099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.040955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.041263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.041537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] PATH_INFO: `/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.042708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef3cd14d18243ef9db7e23dc25fe790)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.044594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.154087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:29.175669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef3cd14d18243ef9db7e23dc25fe790', 'request_id': 'req-5e149135-2de9-4799-b665-370230202c55'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:29.186397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef3cd14d18243ef9db7e23dc25fe790', 'request_id': 'req-5e149135-2de9-4799-b665-370230202c55'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:29.221141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef3cd14d18243ef9db7e23dc25fe790', 'request_id': 'req-5e149135-2de9-4799-b665-370230202c55'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:29.232745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] The token cache is being invalidate because project 0ef3cd14d18243ef9db7e23dc25fe790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:29.233031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:29.233717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5e149135-2de9-4799-b665-370230202c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:29.236561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1078/2159] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:28 2022] DELETE /identity/v3/projects/0ef3cd14d18243ef9db7e23dc25fe790 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:29.241160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.241628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.241721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.260827 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:29.262045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:29.322267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1082/2160] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:29.330331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.330620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.330899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.349535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:29.350766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:29.358650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1079/2161] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:29.363388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.373187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f293d7-6008-40b4-8200-a85b2ce8e8a4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.378161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.378859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.379139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.379415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.380135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:29.380458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.382100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.397016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1083/2162] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:29.398136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.408182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e028a-dd64-46f9-9ffd-9ac33362e068 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.412818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.414059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.414059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.414059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.414744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.416641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.485453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1080/2163] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:29.490552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.500515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4418f-ffc8-4e2b-b0c8-b46ea4ff6853 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.526599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:29.557939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.558599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.558879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.559150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.559843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.561799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.626416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1084/2164] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:29.630329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.640419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aab87-c12a-4479-a876-9648e625d2a8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.645095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.645706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.646016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.646288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.647012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.648585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.659768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1081/2165] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:29.663698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.676247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179216ad-aae3-4cca-a751-1c4651c8815d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.680900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.681588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.681884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.682143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.682951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.684605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ff888-7d12-44c7-9787-a85e3fc3d4f2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.695199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1085/2166] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:29.699298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.709583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb8d7-c01f-4a7b-bb51-728acf8a2205 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.714347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.715298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.715647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.715990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] PATH_INFO: `/v3/projects/c1fe1acf0a3b43a889d9657cdf74993e/users/aa3c46ef65ef4adb80960da7502f8537/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.731590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fe1acf0a3b43a889d9657cdf74993e, user_id=aa3c46ef65ef4adb80960da7502f8537, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.734292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:29.787442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1082/2167] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:29 2022] PUT /identity/v3/projects/c1fe1acf0a3b43a889d9657cdf74993e/users/aa3c46ef65ef4adb80960da7502f8537/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:29.799234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.799583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.799993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.826094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] MFA Rules not processed for user `aa3c46ef65ef4adb80960da7502f8537`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:29.827288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:29.900382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1086/2168] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:29.900519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:29.962143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ce8e5-e0d2-4739-9d1a-67cb49d69ab5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:29.966781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:29.967443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:29.967725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:29.967998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:29.968700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:29.971229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.105075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1083/2169] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:29 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:30.109370 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.120236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed08890-9938-4a57-beee-e6a4ec7a2654 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.176708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62590-cf38-437e-a266-149388770926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.177419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62590-cf38-437e-a266-149388770926 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.177690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62590-cf38-437e-a266-149388770926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.177961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62590-cf38-437e-a266-149388770926 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.178683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb62590-cf38-437e-a266-149388770926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.180592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb62590-cf38-437e-a266-149388770926 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.316967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1087/2170] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:30 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:30.318827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.334378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eca982-4b58-4ffc-9539-ae7edbeeb572 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.353161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.353854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.354138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.354416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.355158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.356769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.369328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1084/2171] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:30.373183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62590-cf38-437e-a266-149388770926 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.388443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62590-cf38-437e-a266-149388770926 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.394506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.395214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.395491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.395776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.396500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.398168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44e92a-826f-4603-b52e-442dc6002a85 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.408575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1088/2172] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:30.412732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.422498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad093e2f-e4fc-4d29-84b9-784c287580f5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.427613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.428478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.428757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.429030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] PATH_INFO: `/v3/projects/1bfbca7c2d9e41e9820299a19afbf766/users/32576255a4a04bd2bb675e4f51de8e88/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.444336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfbca7c2d9e41e9820299a19afbf766, user_id=32576255a4a04bd2bb675e4f51de8e88, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.447123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.542374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:30.547532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1085/2173] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:30 2022] PUT /identity/v3/projects/1bfbca7c2d9e41e9820299a19afbf766/users/32576255a4a04bd2bb675e4f51de8e88/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:30.558079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.558369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.558647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.587177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] MFA Rules not processed for user `32576255a4a04bd2bb675e4f51de8e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:30.588441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:30.592347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:30.655492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1089/2174] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:30.660114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.722975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eec69-78ce-415b-8f22-5bf0b8422a7b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.727611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.728288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.728570 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.728879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.729605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.731541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:30.839260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1086/2175] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:30 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:30.843574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:30.854337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615267fe-bec7-4c89-a526-f372c96a9897 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:30.910536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:30.911195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:30.911475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:30.911747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:30.912423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:30.914349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.038616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1090/2176] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:30 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:31.042678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.054398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ab21b-bf2e-40cb-ae67-9e2bd5597a28 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.058925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.059573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.059853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.060127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.060847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.062471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.072991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1087/2177] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:31.076805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.087203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fecde-5c1e-4172-aba1-dc753093aaad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.091765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.092575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.092855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.093158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] PATH_INFO: `/v3/projects/e4a58ceb3289452c97f4ea818d01a443/users/916ed2f3307b47eb866a0aaa28a56884/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.108309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a58ceb3289452c97f4ea818d01a443, user_id=916ed2f3307b47eb866a0aaa28a56884, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.112062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.161175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.162904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.177169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcda73-f5fd-4a71-b964-93f7c98587fc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.179517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1091/2178] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:31 2022] PUT /identity/v3/projects/e4a58ceb3289452c97f4ea818d01a443/users/916ed2f3307b47eb866a0aaa28a56884/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:31.185922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.235661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.237619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.237903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.238186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] PATH_INFO: `/v3/users/a66fdd9c629842fb89882c9425026b71` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.239712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66fdd9c629842fb89882c9425026b71)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.241659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.249055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421bf19-9915-43bc-8332-04e41ba7852d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.253735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.254393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.254672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.254946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.255666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.257265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.267329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1092/2179] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:31.271369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.281452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f067-dc63-40c9-8c66-898da2997e19 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.286740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.287589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.287875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.288154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] PATH_INFO: `/v3/projects/e4a58ceb3289452c97f4ea818d01a443/users/916ed2f3307b47eb866a0aaa28a56884/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.290145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a58ceb3289452c97f4ea818d01a443, user_id=916ed2f3307b47eb866a0aaa28a56884, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.292897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad19980-66ea-4f91-878f-ad1ebb74a451 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.295162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:31.304537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1093/2180] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:31 2022] PUT /identity/v3/projects/e4a58ceb3289452c97f4ea818d01a443/users/916ed2f3307b47eb866a0aaa28a56884/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:31.314678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.314955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.315276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.342058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] MFA Rules not processed for user `916ed2f3307b47eb866a0aaa28a56884`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:31.343210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.420615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1094/2181] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:31.428513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.428751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66fdd9c629842fb89882c9425026b71'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:31.429901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.443487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66fdd9c629842fb89882c9425026b71'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:31.444502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66fdd9c629842fb89882c9425026b71'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:31.445478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66fdd9c629842fb89882c9425026b71'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:31.458326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-17536a89-3415-42f7-a740-81dec3c666ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66fdd9c629842fb89882c9425026b71'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:31.462283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:31.491288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84982e2-cf8f-4b0e-8883-0ebdfac52d38 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.495172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.495821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.496106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.496383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.498253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.558854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1088/2182] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:31 2022] DELETE /identity/v3/users/a66fdd9c629842fb89882c9425026b71 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:31.561994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.563479 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.563771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.564265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.564828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.577248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1095/2183] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:31.593331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:31.594473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.653790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1089/2184] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:31.664533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:31.666125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:31.703880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:31.725970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dfd1c-5e34-4f26-b3df-cc1952d597db service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:31.776056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:31.776722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:31.776986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:31.777292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:31.786638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:31.788051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:31.798146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1096/2185] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:32.233849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:32.235425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:32.260120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b77359-23ef-4b9a-a361-c6c4cf6c27a9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:32.265101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:32.265984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:32.266268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:32.267504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] PATH_INFO: `/v3/projects/6b5812fce5574d358fa96c7269da086a` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:32.268895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5812fce5574d358fa96c7269da086a)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:32.271009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:32.418182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5812fce5574d358fa96c7269da086a', 'request_id': 'req-76721497-3734-4aa1-9fdc-a4079d6c7d7d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:32.426107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5812fce5574d358fa96c7269da086a', 'request_id': 'req-76721497-3734-4aa1-9fdc-a4079d6c7d7d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:32.493991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5812fce5574d358fa96c7269da086a', 'request_id': 'req-76721497-3734-4aa1-9fdc-a4079d6c7d7d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:32.505738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] The token cache is being invalidate because project 6b5812fce5574d358fa96c7269da086a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:32.506024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:32.506640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:32.508184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1090/2186] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:32 2022] DELETE /identity/v3/projects/6b5812fce5574d358fa96c7269da086a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:34.594724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:34.597387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:34.654072 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520a085-5bc1-4583-b09c-7e9fc235f7a2 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:34.702986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:34.703631 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:34.703991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:34.704324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:34.706397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:34.766196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:34.768169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e17509-3043-4774-af3c-f83f2160d3d7 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:34.777214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1097/2187] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:36.200547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:36.201962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:36.214719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76721497-3734-4aa1-9fdc-a4079d6c7d7d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:36.218317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:36.218980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:36.219264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:36.219544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:36.221272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:36.283284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:36.285447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:36.294915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1091/2188] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:39.067808 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b43457-f37a-4d16-9b35-162b96c3895b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:39.068332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b43457-f37a-4d16-9b35-162b96c3895b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:39.068439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b43457-f37a-4d16-9b35-162b96c3895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:39.074187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:39.086650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-00b43457-f37a-4d16-9b35-162b96c3895b None None] MFA Rules not processed for user `f1c9a7f942c541a287da38a894f1d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:39.087861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00b43457-f37a-4d16-9b35-162b96c3895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:39.146754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1098/2189] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:40.920648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:40.922289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:40.945275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:41.004626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88123c9e-6f83-45ee-b99f-97b40629a7ff service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:41.009577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:41.010476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:41.010755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:41.011031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] PATH_INFO: `/v3/users/aa3c46ef65ef4adb80960da7502f8537` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:41.012273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3c46ef65ef4adb80960da7502f8537)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:41.014277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:41.235761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3c46ef65ef4adb80960da7502f8537'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:41.248154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3c46ef65ef4adb80960da7502f8537'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:41.248843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3c46ef65ef4adb80960da7502f8537'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:41.249474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3c46ef65ef4adb80960da7502f8537'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:41.257316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3c46ef65ef4adb80960da7502f8537'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:41.344502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1092/2190] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:40 2022] DELETE /identity/v3/users/aa3c46ef65ef4adb80960da7502f8537 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:41.346112 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:41.346578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:41.346699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:41.364426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:41.366561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:41.426619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1099/2191] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:41.436529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:41.437935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:41.496496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950f0b-c686-4d5e-b9d8-41eb7107cbc5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:41.500141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:41.500795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:41.501113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:41.501400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:41.512102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:41.513605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:41.526123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1093/2192] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:41.954893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:41.956739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:41.983528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe292d-d39d-452d-a4b1-2915e0ea201f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:41.998969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:41.999882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:42.000125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:42.000402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] PATH_INFO: `/v3/projects/c1fe1acf0a3b43a889d9657cdf74993e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:42.003351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fe1acf0a3b43a889d9657cdf74993e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:42.006000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:42.146787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fe1acf0a3b43a889d9657cdf74993e', 'request_id': 'req-b66ddc7e-74b4-4e02-8e90-f94857ef008b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.155947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fe1acf0a3b43a889d9657cdf74993e', 'request_id': 'req-b66ddc7e-74b4-4e02-8e90-f94857ef008b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.233273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fe1acf0a3b43a889d9657cdf74993e', 'request_id': 'req-b66ddc7e-74b4-4e02-8e90-f94857ef008b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.246544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] The token cache is being invalidate because project c1fe1acf0a3b43a889d9657cdf74993e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:42.250680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.250680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.250680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1100/2193] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:41 2022] DELETE /identity/v3/projects/c1fe1acf0a3b43a889d9657cdf74993e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:42.253727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:42.263629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6aed-4834-4bac-a3d3-da57841f2347 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:42.300135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:42.335365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:42.336506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:42.336667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:42.337067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] PATH_INFO: `/v3/users/32576255a4a04bd2bb675e4f51de8e88` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:42.338475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32576255a4a04bd2bb675e4f51de8e88)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:42.340527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:42.491635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32576255a4a04bd2bb675e4f51de8e88'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.508801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32576255a4a04bd2bb675e4f51de8e88'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.509579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32576255a4a04bd2bb675e4f51de8e88'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.510408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32576255a4a04bd2bb675e4f51de8e88'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.521860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32576255a4a04bd2bb675e4f51de8e88'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:42.585632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1094/2194] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:42 2022] DELETE /identity/v3/users/32576255a4a04bd2bb675e4f51de8e88 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:43.058467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:43.059741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:43.073700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ddc7e-74b4-4e02-8e90-f94857ef008b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:43.110344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:43.139044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:43.140163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:43.140638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:43.140913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] PATH_INFO: `/v3/projects/1bfbca7c2d9e41e9820299a19afbf766` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:43.142152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfbca7c2d9e41e9820299a19afbf766)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:43.145010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:43.267555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfbca7c2d9e41e9820299a19afbf766', 'request_id': 'req-bb21ae2f-49b8-4372-9b9d-887a31d9d306'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.274906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfbca7c2d9e41e9820299a19afbf766', 'request_id': 'req-bb21ae2f-49b8-4372-9b9d-887a31d9d306'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.386352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfbca7c2d9e41e9820299a19afbf766', 'request_id': 'req-bb21ae2f-49b8-4372-9b9d-887a31d9d306'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.407813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] The token cache is being invalidate because project 1bfbca7c2d9e41e9820299a19afbf766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:43.408242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.410113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.412484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1101/2195] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:43 2022] DELETE /identity/v3/projects/1bfbca7c2d9e41e9820299a19afbf766 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:43.416702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:43.418304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:43.448832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d42013-8a70-412d-b9a2-d6b7120dc206 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:43.529957 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:43.530850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:43.531130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:43.531411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] PATH_INFO: `/v3/users/916ed2f3307b47eb866a0aaa28a56884` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:43.532612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916ed2f3307b47eb866a0aaa28a56884)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:43.534988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:43.781165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916ed2f3307b47eb866a0aaa28a56884'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.786991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:43.796024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916ed2f3307b47eb866a0aaa28a56884'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.804355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916ed2f3307b47eb866a0aaa28a56884'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.804355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916ed2f3307b47eb866a0aaa28a56884'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.806887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-42b5a254-f77f-473b-a812-a218721be324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916ed2f3307b47eb866a0aaa28a56884'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:43.911895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:43.913359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:43.915579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1095/2196] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:43 2022] DELETE /identity/v3/users/916ed2f3307b47eb866a0aaa28a56884 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:43.929255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ae2f-49b8-4372-9b9d-887a31d9d306 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:43.998304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:43.999283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:43.999667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.000048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] PATH_INFO: `/v3/users/b3a84a70c193447d8c5ef91aad92a449` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.001983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a84a70c193447d8c5ef91aad92a449)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:44.004594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:44.318877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a84a70c193447d8c5ef91aad92a449'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.326714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a84a70c193447d8c5ef91aad92a449'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.340834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a84a70c193447d8c5ef91aad92a449'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.342170 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a84a70c193447d8c5ef91aad92a449'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.409818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a84a70c193447d8c5ef91aad92a449'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.416591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1102/2197] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:43 2022] DELETE /identity/v3/users/b3a84a70c193447d8c5ef91aad92a449 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:44.421072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.421560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.421655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.445441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:44.446686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:44.518215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1096/2198] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:44.525899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:44.527635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:44.531263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:44.532917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:44.546216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5d79-d880-429d-b463-5d7d59be1751 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:44.552013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:44.553151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.553510 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.554140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] PATH_INFO: `/v3/projects/e4a58ceb3289452c97f4ea818d01a443` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.555460 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a58ceb3289452c97f4ea818d01a443)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:44.557647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:44.604508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:44.605467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6dfa8-7a51-4c0d-be2c-4320a452ad0c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:44.609458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:44.610696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.610985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.611257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.621917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:44.623388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67ce4a-d3f6-4391-921f-04dba3ec2261 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:44.635719 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1097/2199] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:44.678786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a58ceb3289452c97f4ea818d01a443', 'request_id': 'req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.686298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a58ceb3289452c97f4ea818d01a443', 'request_id': 'req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.763046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a58ceb3289452c97f4ea818d01a443', 'request_id': 'req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.774685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] The token cache is being invalidate because project e4a58ceb3289452c97f4ea818d01a443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:44.775068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.775799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-409d054e-9b7d-4c8d-a305-22e5c9cff0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:44.777718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1103/2200] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:44 2022] DELETE /identity/v3/projects/e4a58ceb3289452c97f4ea818d01a443 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:44.782648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.782987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.783271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.803853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:44.805124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:44.872532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1098/2201] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:44.880652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.881085 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.881424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.901789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:44.903177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:44.915763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1104/2202] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:44.918962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:44.948232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93495a7c-76f7-4329-b3db-6b59da111dc8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:44.957396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:44.957396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:44.957396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:44.957396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:44.957906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:44.958146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:44.960467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:44.976447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1099/2203] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:44.980851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:44.994766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe137c-9381-4da2-83a7-f9a34b556ef1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.001272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.005660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.006134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.006860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.007665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.009718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.124696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1105/2204] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:45.131680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.134460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:45.140072 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8960-67bd-49c1-bedc-132686f68afd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.164877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.166923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:45.223117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.223796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.224079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.224362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.225085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.227006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.257621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67beae8-71e9-4721-a490-b76d11ecd2ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.262452 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.263311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.263594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.263872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] PATH_INFO: `/v3/projects/48c727a7e3454f26a810b5bf457023f9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.265146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c727a7e3454f26a810b5bf457023f9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.267062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.312009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1100/2205] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:45.319443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.329855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96bca5-9982-4de9-b777-3ba3639ad224 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.334707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.335368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.335651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.335924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.336645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.338306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.349000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1101/2206] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:45.352807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.363173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a819d-8862-41e4-beec-077882689e83 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.367929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.368589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.368877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.369597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.369919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.371525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.384025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1102/2207] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:45.385385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c727a7e3454f26a810b5bf457023f9', 'request_id': 'req-d85890dc-3ffe-4c12-9cf8-2337587b7b60'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:45.387908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.392888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c727a7e3454f26a810b5bf457023f9', 'request_id': 'req-d85890dc-3ffe-4c12-9cf8-2337587b7b60'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:45.398478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f746ad-3ab6-4168-ac04-4ebac943d4f6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.455084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.456526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.456808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.457190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] PATH_INFO: `/v3/projects/77436c23422442e4a33f2fd38d8979c6/users/af43184875884b9b8f608203914c89ab/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.466590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c727a7e3454f26a810b5bf457023f9', 'request_id': 'req-d85890dc-3ffe-4c12-9cf8-2337587b7b60'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:45.474337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77436c23422442e4a33f2fd38d8979c6, user_id=af43184875884b9b8f608203914c89ab, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.477593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.478926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] The token cache is being invalidate because project 48c727a7e3454f26a810b5bf457023f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:45.479527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:45.480777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d85890dc-3ffe-4c12-9cf8-2337587b7b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:45.482921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1106/2208] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:45 2022] DELETE /identity/v3/projects/48c727a7e3454f26a810b5bf457023f9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:45.487653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719188c-cf03-464b-8c96-8e45e4623118 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.487946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719188c-cf03-464b-8c96-8e45e4623118 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.488226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719188c-cf03-464b-8c96-8e45e4623118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.506044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3719188c-cf03-464b-8c96-8e45e4623118 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:45.507229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3719188c-cf03-464b-8c96-8e45e4623118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:45.515128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1107/2209] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:45.523989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.524278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.524581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.542873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:45.544063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:45.552679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1108/2210] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:45.556818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.566589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61d18c-2747-4add-ba31-ef2cadb5be08 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.571251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.571879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.572164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.572442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.573228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:45.573536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.575192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84308c-23cd-43e0-905a-644731fe7991 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.596350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1103/2211] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:45 2022] PUT /identity/v3/projects/77436c23422442e4a33f2fd38d8979c6/users/af43184875884b9b8f608203914c89ab/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:45.596633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1109/2212] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:51:45.596728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.597978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.598265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.598542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.605979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:45.631670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] MFA Rules not processed for user `af43184875884b9b8f608203914c89ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:45.632920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:45.673883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffaabe0-286b-4d88-8e04-cc7e94262389 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.680192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.680695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.681119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.682256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.682395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.684462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.704847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1110/2213] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:45.717877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.719285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:45.777886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5839f11-eac2-45c6-8ec1-4cc0e563d45e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.781547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.783031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.783031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.783031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.789484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1104/2214] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:45.793577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.794142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:45.795022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.808477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1111/2215] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:45.871365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a1f2e-fa10-4828-8250-2517319c9ea7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:45.877251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:45.878619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:45.878870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:45.879076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:45.879785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:45.881860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:45.998150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1105/2216] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:45 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:46.002399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.013669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2172566-7133-4d7f-be6e-c8b25aca87e9 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.018164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.018948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.019324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.019606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.020333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.022025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.035951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1112/2217] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:46.036102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.036690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.054963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:46.055410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1c766-9c27-431a-b418-dbf40fa607cb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.055504 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1f20e-500e-4079-906b-cc7bdd23bf41 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.055609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.055714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.055919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.056200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.056932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.058228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.058553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.059120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.059408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.059683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] PATH_INFO: `/v3/users/4f4d80e277a14e58bc31da5bd3fbed28` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.061145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f4d80e277a14e58bc31da5bd3fbed28)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.063125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.069759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1113/2218] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:46.073946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.087253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372386-ebdd-4e23-89e9-8e0b1d2e4fe3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.089693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.090540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.090820 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.091090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] PATH_INFO: `/v3/projects/fa16115e6ada4f16874b4f48730bec7f/users/8171d4ef9f644866bb5747a6c31fa5cf/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.107893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa16115e6ada4f16874b4f48730bec7f, user_id=8171d4ef9f644866bb5747a6c31fa5cf, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.109418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409ec48-6ca7-4c2e-99eb-d7e1f5819cc7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.145750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:46.240372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1114/2219] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:46 2022] PUT /identity/v3/projects/fa16115e6ada4f16874b4f48730bec7f/users/8171d4ef9f644866bb5747a6c31fa5cf/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:46.245101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.245397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.245675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.263275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:46.264488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:46.303113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f4d80e277a14e58bc31da5bd3fbed28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.316231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f4d80e277a14e58bc31da5bd3fbed28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.317310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f4d80e277a14e58bc31da5bd3fbed28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.318032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f4d80e277a14e58bc31da5bd3fbed28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.324074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1115/2220] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:46.325664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f4d80e277a14e58bc31da5bd3fbed28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.333864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.336260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:46.394563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecdc57-1432-4cb9-a2c2-a77100942323 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.398336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.398980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.399293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.399566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.409475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1106/2221] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:46 2022] DELETE /identity/v3/users/4f4d80e277a14e58bc31da5bd3fbed28 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:46.413611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.413611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.422138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1116/2222] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:46.541021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:46.724747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:46.726227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:46.781684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:46.806526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd25f-643b-4877-b69c-f6fe256083dd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:46.811203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:46.812007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:46.812286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:46.812566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] PATH_INFO: `/v3/projects/a6a4088a03664df295e7e6a4aac6f79e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:46.813786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a4088a03664df295e7e6a4aac6f79e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:46.816276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:46.960096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a4088a03664df295e7e6a4aac6f79e', 'request_id': 'req-f28147ab-cbad-484d-a92a-e9931d614d63'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:46.971126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a4088a03664df295e7e6a4aac6f79e', 'request_id': 'req-f28147ab-cbad-484d-a92a-e9931d614d63'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.071936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a4088a03664df295e7e6a4aac6f79e', 'request_id': 'req-f28147ab-cbad-484d-a92a-e9931d614d63'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.085172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] The token cache is being invalidate because project a6a4088a03664df295e7e6a4aac6f79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:47.085490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.086135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.087792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1107/2223] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:46 2022] DELETE /identity/v3/projects/a6a4088a03664df295e7e6a4aac6f79e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:47.091699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:47.101405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e44d7-40a1-4157-93c3-1cc8a1fdf6f0 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:47.157521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:47.158396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:47.158680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:47.158952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] PATH_INFO: `/v3/users/3427fe2a85f944e597a1e58ff4849bb2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:47.160168 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3427fe2a85f944e597a1e58ff4849bb2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:47.162175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:47.382835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3427fe2a85f944e597a1e58ff4849bb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.389701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3427fe2a85f944e597a1e58ff4849bb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.398943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:47.404381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3427fe2a85f944e597a1e58ff4849bb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.405297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3427fe2a85f944e597a1e58ff4849bb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.523297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3427fe2a85f944e597a1e58ff4849bb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:47.529728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1117/2224] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:47 2022] DELETE /identity/v3/users/3427fe2a85f944e597a1e58ff4849bb2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:47.810745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:47.812281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:47.827565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28147ab-cbad-484d-a92a-e9931d614d63 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:47.889164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:47.889964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:47.890247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:47.890525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] PATH_INFO: `/v3/projects/5462e375e4b14aeba6059b461442d4d8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:47.891734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5462e375e4b14aeba6059b461442d4d8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:47.893930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:48.024671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5462e375e4b14aeba6059b461442d4d8', 'request_id': 'req-61287c9e-daf7-43a1-935a-f9c8c0791934'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:48.032521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5462e375e4b14aeba6059b461442d4d8', 'request_id': 'req-61287c9e-daf7-43a1-935a-f9c8c0791934'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:48.140185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5462e375e4b14aeba6059b461442d4d8', 'request_id': 'req-61287c9e-daf7-43a1-935a-f9c8c0791934'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:48.148697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:48.152814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] The token cache is being invalidate because project 5462e375e4b14aeba6059b461442d4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:48.153211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:48.153913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61287c9e-daf7-43a1-935a-f9c8c0791934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:48.155542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1108/2225] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:47 2022] DELETE /identity/v3/projects/5462e375e4b14aeba6059b461442d4d8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:49.051882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:49.054372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:49.068703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58a0c-841d-4023-a134-1302582a7874 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:49.166266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:49.167090 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:49.167375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:49.167650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] PATH_INFO: `/v3/users/af43184875884b9b8f608203914c89ab` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:49.169587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af43184875884b9b8f608203914c89ab)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:49.172648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:49.367876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af43184875884b9b8f608203914c89ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:49.375019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af43184875884b9b8f608203914c89ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:49.388653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af43184875884b9b8f608203914c89ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:49.389425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af43184875884b9b8f608203914c89ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:49.427167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af43184875884b9b8f608203914c89ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:49.433884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1118/2226] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:51:49 2022] DELETE /identity/v3/users/af43184875884b9b8f608203914c89ab => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:49.438523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:49.439009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:49.442889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:49.459032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:49.460389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:49.522407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1109/2227] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:49.532156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:49.533636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:49.571540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:49.609496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0551d2f-7096-4a78-b868-2f7c62e56d3b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:49.615397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:49.615758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:49.616144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:49.616431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:49.626798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:49.628857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813eecf-73e5-498a-b9f8-e4319c006e7b service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:49.639179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1119/2228] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:51:50.215870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:50.217389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:50.236717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f235-1d38-453c-9c65-423a77133720 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:50.243679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:50.244496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.244783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.245091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] PATH_INFO: `/v3/projects/77436c23422442e4a33f2fd38d8979c6` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.246814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77436c23422442e4a33f2fd38d8979c6)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:50.249415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:50.392543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77436c23422442e4a33f2fd38d8979c6', 'request_id': 'req-3aca1598-8723-4a5b-b49d-852827f658b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:50.400411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77436c23422442e4a33f2fd38d8979c6', 'request_id': 'req-3aca1598-8723-4a5b-b49d-852827f658b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:50.472297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77436c23422442e4a33f2fd38d8979c6', 'request_id': 'req-3aca1598-8723-4a5b-b49d-852827f658b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:50.485077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] The token cache is being invalidate because project 77436c23422442e4a33f2fd38d8979c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:51:50.485399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:50.486216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3aca1598-8723-4a5b-b49d-852827f658b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:51:50.492419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1110/2229] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:51:50 2022] DELETE /identity/v3/projects/77436c23422442e4a33f2fd38d8979c6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:50.495269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.496156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.496596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.517536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:50.519284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:50.592654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1120/2230] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:50.601557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.601848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.602191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.623478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:50.625268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:50.636509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1111/2231] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:50.640760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:50.664794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a56c1-1e18-4e63-a702-6cd1e4bf2b6f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:50.670634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:50.671196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.671661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.672247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.673477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:51:50.673959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:50.677143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:50.695549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1121/2232] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:51:50.695936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:50.711613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f300b0-7d93-4170-95bf-776614d0c677 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:50.726110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:50.727374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.727825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.728413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.729376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:50.731523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:50.870550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1112/2233] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:50 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:51:50.878357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:50.888356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a03a5-76a7-4e1a-b91e-ff0a16f0be02 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:50.960299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:50.961339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:50.961628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:50.961984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:50.962744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:50.964699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:51.062574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1122/2234] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:50 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:51.066564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:51.085835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6d29e-aea6-4901-8bda-b47f3bdafba2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:51.092171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:51.097383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:51.097679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:51.097960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:51.098743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:51.100823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:51.119177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1113/2235] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:51.120581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:51.133886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b7ea1-8828-428b-a689-d47fcffeac29 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:51.140006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:51.141265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:51.141565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:51.141849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:51.142636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:51.145614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42319-51b1-43ea-b59c-9be211af2001 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:51.159334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1123/2236] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:51.163676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:51.183448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f6d36-2742-456b-b7fa-a0d61183c302 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:51.188688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:51.189859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:51.190399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:51.190854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] PATH_INFO: `/v3/projects/67a506c215d945909c3fb7dd7f9290ba/users/7b331f1fb94c4f17af15ddac69ab57ab/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:51.225130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a506c215d945909c3fb7dd7f9290ba, user_id=7b331f1fb94c4f17af15ddac69ab57ab, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:51.227855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:51.306678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1114/2237] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:51 2022] PUT /identity/v3/projects/67a506c215d945909c3fb7dd7f9290ba/users/7b331f1fb94c4f17af15ddac69ab57ab/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:51:51.372698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b150-44ba-4150-95b7-ce94f94bdee1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:51.372863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b150-44ba-4150-95b7-ce94f94bdee1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:51.373142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9b150-44ba-4150-95b7-ce94f94bdee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:51.403708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-78e9b150-44ba-4150-95b7-ce94f94bdee1 None None] MFA Rules not processed for user `7b331f1fb94c4f17af15ddac69ab57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:51.405209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78e9b150-44ba-4150-95b7-ce94f94bdee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:51.448255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:51.507357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1124/2238] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:51.514961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:51.517438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:51.546874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:51.620140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb94f02-8d89-43df-83af-87f8ff27da72 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:51.624845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:51.625791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:51.626077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:51.626346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:51.638137 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:51.643281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:51.658129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1115/2239] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:51:52.807122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:52.807696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:52.807776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:52.836444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] MFA Rules not processed for user `8171d4ef9f644866bb5747a6c31fa5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:52.838599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:52.920335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1125/2240] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:51:52.925864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:52.927507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:53.004931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d7eaa-2b46-418d-893b-26ea8f8ab597 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.010142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.010820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.011099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.011371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.012106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.014194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.095336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1116/2241] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:51:52 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:53.099852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.114757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374d37-52de-4e3d-b041-6b2f77ca3c30 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.174286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.175880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.176235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.176596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.177426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.181124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.277079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1126/2242] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:51:53 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:51:53.280593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.292641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2a843-05c4-43ac-9821-a6175f0d5e15 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.298392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.300823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.300823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.301027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.301929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.303629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.317760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1117/2243] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:51:53.322211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.333959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09266a6-6c9b-42d9-8095-7618106a548a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.338953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.339867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.340155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.340436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] PATH_INFO: `/v3/projects/18a6f76c9d664670b52552f99558398f/users/ee4ca294ed084d12bceed71bef4d1615/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.359480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a6f76c9d664670b52552f99558398f, user_id=ee4ca294ed084d12bceed71bef4d1615, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.363133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046956-a872-4596-aad9-7e48afe2b126 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.466980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1127/2244] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:53 2022] PUT /identity/v3/projects/18a6f76c9d664670b52552f99558398f/users/ee4ca294ed084d12bceed71bef4d1615/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:53.470775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.483057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c8cfe-0ef6-4fba-b76c-6fcd012fddf1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.486572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.486914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.487294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.509746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] MFA Rules not processed for user `7b331f1fb94c4f17af15ddac69ab57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:51:53.511325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:53.533086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:51:53.566341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.567572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.567860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.568134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.568917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.570866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.584695 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1118/2245] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:51:53.589452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.601722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5968a0-51d9-472a-8e05-7efc9cf32f29 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:53.606790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:53.607699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:53.608047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:53.608323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] PATH_INFO: `/v3/projects/18a6f76c9d664670b52552f99558398f/users/ee4ca294ed084d12bceed71bef4d1615/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:53.610433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a6f76c9d664670b52552f99558398f, user_id=ee4ca294ed084d12bceed71bef4d1615, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:53.613245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d1ae-1f75-4276-88a6-d190a726dba5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:53.620052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1128/2246] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:51:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:51:53.633100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1119/2247] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:51:53 2022] PUT /identity/v3/projects/18a6f76c9d664670b52552f99558398f/users/ee4ca294ed084d12bceed71bef4d1615/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:51:53.943560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:51:53.945773 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:54.043499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6339-4f75-4f8e-b035-43ee1bd768fc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:51:54.160375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:51:54.161196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:51:54.161482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:51:54.173105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:51:54.176680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:51:54.190425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:51:54.192759 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:51:54.217892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1129/2248] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:51:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:52:01.411138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae51ba-09cb-439b-8e20-fe86f8b61bc8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:01.411649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae51ba-09cb-439b-8e20-fe86f8b61bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:01.411762 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae51ba-09cb-439b-8e20-fe86f8b61bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:01.441596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-85ae51ba-09cb-439b-8e20-fe86f8b61bc8 None None] MFA Rules not processed for user `ee4ca294ed084d12bceed71bef4d1615`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:01.442799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-85ae51ba-09cb-439b-8e20-fe86f8b61bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:01.451527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:01.512358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1120/2249] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:01.524337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:01.525694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:01.587823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144b2e0-6656-4ee3-90df-3fdd732b5ed3 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:01.592074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:01.592818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:01.593230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:01.593603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:01.605913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:01.657614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:01.659315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:01.668426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1130/2250] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:52:01.917986 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7eab4-b131-4083-a2a4-bee9e3a51de3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:01.918280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7eab4-b131-4083-a2a4-bee9e3a51de3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:01.918554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7eab4-b131-4083-a2a4-bee9e3a51de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:01.938770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f1e7eab4-b131-4083-a2a4-bee9e3a51de3 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:01.941358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f1e7eab4-b131-4083-a2a4-bee9e3a51de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:01.996937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1121/2251] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:02.038998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:02.040419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:02.113897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe94ac8-2c32-4ca4-acd9-2b5f8aca72e8 service glance] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:02.118926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:02.119573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:02.119857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:02.120132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:02.129375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:02.130963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a31675-1717-4799-94cb-f027641e7928 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:02.139137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1131/2252] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:11.990169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=50 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:12.135795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=54 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:13.254996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:13.255579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:13.255579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:13.274392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] MFA Rules not processed for user `8171d4ef9f644866bb5747a6c31fa5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:13.275601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:13.282659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1122/2253] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:15.540929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb72f4-c915-4c33-8962-33ffcf0d20ec None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:15.540929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb72f4-c915-4c33-8962-33ffcf0d20ec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:15.540929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb72f4-c915-4c33-8962-33ffcf0d20ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:15.553665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f2cb72f4-c915-4c33-8962-33ffcf0d20ec None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:15.555020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f2cb72f4-c915-4c33-8962-33ffcf0d20ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:15.562605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1132/2254] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:52:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:15.575179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:15.587548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73484fbf-0295-44fc-a5c2-8bce7ea4d673 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:15.592498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:15.593174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:15.593458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:15.593732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:15.604302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:15.605969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:15.616126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1123/2255] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:16.562229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3972-3120-4b3c-babe-9d71983b10af None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:16.562669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3972-3120-4b3c-babe-9d71983b10af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:16.565113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3972-3120-4b3c-babe-9d71983b10af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:16.580534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-565f3972-3120-4b3c-babe-9d71983b10af None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:16.581772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-565f3972-3120-4b3c-babe-9d71983b10af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:16.588993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1133/2256] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 12:52:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:16.603099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:16.614979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f622f-6697-4d49-b5bc-a4089888a3ae service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:16.620370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:16.621298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:16.621864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:16.622305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:16.632717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:16.634706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:16.643685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1124/2257] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:16.916496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d876de5-a17a-4535-9c83-2f63c250a6d3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:16.916786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d876de5-a17a-4535-9c83-2f63c250a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:16.921798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d876de5-a17a-4535-9c83-2f63c250a6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:16.935087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0d876de5-a17a-4535-9c83-2f63c250a6d3 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:16.936297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0d876de5-a17a-4535-9c83-2f63c250a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:16.943513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1134/2258] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:52:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:16.956600 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:16.967653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc5f-d7e6-4178-8b9e-2ef302c9028b service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:16.975046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:16.976377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:16.976671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:16.979177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:16.987856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:16.989484 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:16.998214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1125/2259] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:17.355857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96352d2-ffe4-42c8-b24d-080db9013e4d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:17.356409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96352d2-ffe4-42c8-b24d-080db9013e4d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:17.357562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96352d2-ffe4-42c8-b24d-080db9013e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:17.378623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d96352d2-ffe4-42c8-b24d-080db9013e4d None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:17.380326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d96352d2-ffe4-42c8-b24d-080db9013e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:17.397410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1135/2260] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:52:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:17.418785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:17.435615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e331f2d-9ab7-408f-996f-da7bdf463382 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:17.451895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:17.452584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:17.453174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:17.453459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:17.467237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:17.468949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:17.480233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1126/2261] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:18.117741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:18.118225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:18.118330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:18.147383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:18.150995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:18.165987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1136/2262] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:52:18.177541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:18.198639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211798b6-ab49-45e4-9a19-2f545534ab28 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:18.198639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:18.198639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:18.201450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:18.201688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:18.212424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:18.214215 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c09d-ed9c-49b3-8593-b3de8b691bd6 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:18.228824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1127/2263] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:52:21.017438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:21.102322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22623e9-bbfe-4d9e-ae05-b456f9c941b2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:21.109586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:21.110621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:21.110911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:21.111188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] PATH_INFO: `/v3/users/7b331f1fb94c4f17af15ddac69ab57ab` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:21.112419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b331f1fb94c4f17af15ddac69ab57ab)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:21.122603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:21.321392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b331f1fb94c4f17af15ddac69ab57ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:21.328109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b331f1fb94c4f17af15ddac69ab57ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:21.341176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b331f1fb94c4f17af15ddac69ab57ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:21.348138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b331f1fb94c4f17af15ddac69ab57ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:21.426199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b331f1fb94c4f17af15ddac69ab57ab'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:21.433998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1137/2264] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:52:21 2022] DELETE /identity/v3/users/7b331f1fb94c4f17af15ddac69ab57ab => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:52:21.438380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:21.438925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:21.439069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:21.464071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:21.464071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:21.524164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1128/2265] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:52:21.534016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:21.536292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:21.598893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:21.603993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02883d32-73cc-4748-a064-6ef2ddc974fd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:21.607901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:21.608564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:21.608889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:21.609142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:21.619383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:21.620832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3c540-ce40-44c9-bf0e-3b04055c4366 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:21.632967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1138/2266] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:52:21.899168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:21.900749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:21.916626 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b49b20-1cb2-455a-99c2-f71b08d363bc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:21.921503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:21.922395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:21.922685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:21.922939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] PATH_INFO: `/v3/projects/67a506c215d945909c3fb7dd7f9290ba` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:21.924142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a506c215d945909c3fb7dd7f9290ba)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:21.927049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:22.027745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a506c215d945909c3fb7dd7f9290ba', 'request_id': 'req-6a034cdf-a43c-4040-a36a-b6806159749c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:22.034487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a506c215d945909c3fb7dd7f9290ba', 'request_id': 'req-6a034cdf-a43c-4040-a36a-b6806159749c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:22.094345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a506c215d945909c3fb7dd7f9290ba', 'request_id': 'req-6a034cdf-a43c-4040-a36a-b6806159749c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:22.106911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] The token cache is being invalidate because project 67a506c215d945909c3fb7dd7f9290ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:52:22.111335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:22.111335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6a034cdf-a43c-4040-a36a-b6806159749c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:52:22.111335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1129/2267] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:52:21 2022] DELETE /identity/v3/projects/67a506c215d945909c3fb7dd7f9290ba => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:52:22.117109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.117109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.117749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.140887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:22.142161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:22.219221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1139/2268] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:22.227533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.227907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.228341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.265590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:22.266931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:22.289397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1130/2269] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:22.293527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:22.329810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98e4c-a8cb-4152-be22-3cfe4dc086ea None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:22.341702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:22.343046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.343619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.343895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.344659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:52:22.345280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:22.346904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:22.377137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1140/2270] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:52:22.377915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:22.385378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:22.391302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873748-b3c9-49a5-8c92-19169f68b0f0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:22.404060 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:22.404779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.408030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.408338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.409131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:22.411611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:22.499248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1131/2271] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:52:22 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:52:22.504893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:22.518831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38329bd-9cb6-45c0-b4aa-b104b288d164 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:22.579034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:22.579721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.580066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.580408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.581245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:22.583866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:22.928322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1141/2272] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:52:22 2022] POST /identity/v3/users => generated 286 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:52:22.930077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:22.940668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028cfda-ce2d-49f3-a7e6-fd730684caf1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:22.945676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:22.951286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:22.951286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:22.951286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:22.951286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:22.951286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:22.963707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1132/2273] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:52:22.967946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:22.996841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9b2fb-6b16-4866-a651-f0794c2f29c7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:23.018724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:23.021909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:23.022109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:23.022400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:23.023155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:23.026098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f47f1-25b6-43a1-bf16-ecebc5712236 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:23.061066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1142/2274] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:52:23.064470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:23.078902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68693e1-b2c8-4519-aa26-042053fdd4be admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:23.091182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:23.092467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:23.092903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:23.093907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] PATH_INFO: `/v3/projects/73deeaa0bcd7498eae06c39dfc0e2e3b/users/8431eaa3717a4f798db80c9c87832273/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:23.131345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73deeaa0bcd7498eae06c39dfc0e2e3b, user_id=8431eaa3717a4f798db80c9c87832273, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:23.134877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:23.266337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1133/2275] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:52:23 2022] PUT /identity/v3/projects/73deeaa0bcd7498eae06c39dfc0e2e3b/users/8431eaa3717a4f798db80c9c87832273/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:52:23.272304 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d53f-0e6b-47eb-95d0-c38526aafcd9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:23.272853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d53f-0e6b-47eb-95d0-c38526aafcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:23.272942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d53f-0e6b-47eb-95d0-c38526aafcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:23.309685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7c00d53f-0e6b-47eb-95d0-c38526aafcd9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:23.309685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7c00d53f-0e6b-47eb-95d0-c38526aafcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:23.369435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:23.383366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1143/2276] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:23.393226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:23.402047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:23.473401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f6dce-701b-44e1-a7b6-e92d5cbbc5e8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:23.477608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:23.478349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:23.478604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:23.478891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:23.510482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:23.512676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:23.535357 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1134/2277] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:52:30.017962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.018507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.018507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.047351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] MFA Rules not processed for user `8431eaa3717a4f798db80c9c87832273`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:30.048568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:30.121020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1144/2278] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:30.125749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.136505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fefd38-8a06-441f-8ceb-ea9e8eaa05b4 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.140966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.141644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.141920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.142191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.142884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.145875 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.240449 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1135/2279] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:52:30 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:52:30.244756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.256528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53d11e-692e-4ce1-9ef4-080192a7db20 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.323755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.324426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.324706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.324976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.325702 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.327609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.455084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1145/2280] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:52:30 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:52:30.459710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.470506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674106e2-640b-40ea-a82f-6f6cbad75079 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.475697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.481977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.481977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.481977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.481977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.481977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.493685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1136/2281] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:52:30.497681 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.508521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7a3e2-8cdb-4bb3-9e1a-a878cf2db76c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.513270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.514219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.514507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.514783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] PATH_INFO: `/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf/users/8ef5ec397a8a44fda3644476b2f66586/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.531067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b483ffd8ef6e41708af269ea8ad3d6bf, user_id=8ef5ec397a8a44fda3644476b2f66586, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.533862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.621548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1146/2282] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:52:30 2022] PUT /identity/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf/users/8ef5ec397a8a44fda3644476b2f66586/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:52:30.625576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.636276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893594c1-231d-4ea3-b9a1-c43965c67249 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.704145 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.704823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.705229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.705536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.706262 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.708143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c00f64-dda3-4181-ac69-2ab7f449aa6a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.721443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1137/2283] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:52:30.723967 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:30.735595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045b6a-23ec-4ae7-94af-9a3d21541631 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:30.741286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:30.742178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:30.742446 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:30.742721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] PATH_INFO: `/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf/users/8ef5ec397a8a44fda3644476b2f66586/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:30.744690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b483ffd8ef6e41708af269ea8ad3d6bf, user_id=8ef5ec397a8a44fda3644476b2f66586, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:30.747461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:30.758704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1147/2284] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:52:30 2022] PUT /identity/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf/users/8ef5ec397a8a44fda3644476b2f66586/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:52:36.987342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3011e10-e605-4e11-be17-86bd6f2e2c5b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:36.987859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3011e10-e605-4e11-be17-86bd6f2e2c5b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:36.988499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3011e10-e605-4e11-be17-86bd6f2e2c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:37.014050 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:37.017335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a3011e10-e605-4e11-be17-86bd6f2e2c5b None None] MFA Rules not processed for user `8ef5ec397a8a44fda3644476b2f66586`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:37.018563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a3011e10-e605-4e11-be17-86bd6f2e2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:37.104143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1138/2285] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:37.119546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:37.121531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:37.172054 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:37.211477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71493352-b8cd-45ec-adc4-e5c0d61a6ffa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:37.217021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:37.217711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:37.217994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:37.218270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:37.291695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:37.293311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377b0d-337e-4105-a156-07f2236fbaeb service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:37.303044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1148/2286] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:52:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:52:39.827550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878bbc-1418-4b9d-bde3-e1de3e0ec146 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:39.829818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878bbc-1418-4b9d-bde3-e1de3e0ec146 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:39.829818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878bbc-1418-4b9d-bde3-e1de3e0ec146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:39.845715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-66878bbc-1418-4b9d-bde3-e1de3e0ec146 None None] MFA Rules not processed for user `8431eaa3717a4f798db80c9c87832273`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:39.846901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-66878bbc-1418-4b9d-bde3-e1de3e0ec146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:39.854961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1139/2287] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:47.298100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=39 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:47.525386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:47.525679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:47.525955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:47.543999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] MFA Rules not processed for user `628e662a3b49438f9930a2d160658b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:47.545241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:47.613352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1149/2288] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:52:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:49.840238 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=31 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:57.607248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=25 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:52:59.590473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:59.590970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:59.591220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:59.609391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:52:59.610595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:59.667560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1140/2289] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:52:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:52:59.683655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:52:59.683655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:52:59.758710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e30c2-dd58-47b4-b02a-7f498b38df70 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:52:59.763836 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:52:59.767549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:52:59.767549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:52:59.767549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:52:59.773912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:52:59.775498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f2510-4b3e-4af8-a974-105e4f7a76f8 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:52:59.783780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1150/2290] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 12:53:02.277460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:02.336210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97f3bb-3adc-4143-9da3-f9eb0745631e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:02.339939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:02.340915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:02.340915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:02.341246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:02.343065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:02.412138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:02.412380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6754ed-717b-4e68-943d-bac584298f91 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:02.424379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1141/2291] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:09.780494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=32 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:53:12.420013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=75 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:53:23.085929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.090461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.090461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.090461 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1151/2292] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:53:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:53:23.095495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843e21-a7c1-4138-a341-bced07b7e005 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.096028 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843e21-a7c1-4138-a341-bced07b7e005 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.096168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d843e21-a7c1-4138-a341-bced07b7e005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.114047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4d843e21-a7c1-4138-a341-bced07b7e005 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:53:23.115350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4d843e21-a7c1-4138-a341-bced07b7e005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:23.126270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1142/2293] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:53:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:53:23.142251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:23.143969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:23.204516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce307181-8317-4cf0-93ee-834ce797cbfd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:23.208097 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:23.208744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.209051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.209346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.220529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:23.221924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca7cb6-9842-4e63-b4e4-6b2023f03483 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:23.234003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1152/2294] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:23.453241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26342430-7240-48a4-8926-a415f07d0ddf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.453528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26342430-7240-48a4-8926-a415f07d0ddf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.453858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26342430-7240-48a4-8926-a415f07d0ddf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.455388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1143/2295] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:53:23.460426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d429c-652e-44ac-80e4-ffc98a8f0017 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.460797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d429c-652e-44ac-80e4-ffc98a8f0017 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.461191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d429c-652e-44ac-80e4-ffc98a8f0017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.479882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-644d429c-652e-44ac-80e4-ffc98a8f0017 None None] MFA Rules not processed for user `246c8e2b8aec415496a87067875975a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:53:23.481015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-644d429c-652e-44ac-80e4-ffc98a8f0017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:23.487564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1153/2296] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:53:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:53:23.499828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26342430-7240-48a4-8926-a415f07d0ddf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:23.513803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26342430-7240-48a4-8926-a415f07d0ddf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:23.517621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:23.518269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.518608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.518822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.526730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:23.528153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea900e-5ef5-48bc-a5f2-a008f4691f9d service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:23.536617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1144/2297] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:23.627286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ffe0-5117-4489-935a-ccb24f50a2ea None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.627598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ffe0-5117-4489-935a-ccb24f50a2ea None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.627872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ffe0-5117-4489-935a-ccb24f50a2ea None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.629360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1154/2298] 10.222.0.47 () {54 vars in 921 bytes} [Sat Jul 16 12:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 12:53:23.635380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3efa-5b2f-48c3-8028-248a6f7b46d4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.635665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3efa-5b2f-48c3-8028-248a6f7b46d4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.635941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3efa-5b2f-48c3-8028-248a6f7b46d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.653976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cb6b3efa-5b2f-48c3-8028-248a6f7b46d4 None None] MFA Rules not processed for user `0faa6a54416f420bbdec92832d404efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:53:23.655168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cb6b3efa-5b2f-48c3-8028-248a6f7b46d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:23.664377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1145/2299] 10.222.0.47 () {58 vars in 1018 bytes} [Sat Jul 16 12:53:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:53:23.675466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0ffe0-5117-4489-935a-ccb24f50a2ea None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:23.687229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0ffe0-5117-4489-935a-ccb24f50a2ea None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:23.690791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:23.691416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:23.691698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:23.691975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:23.702251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:23.703650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:23.714290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1155/2300] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:27.571723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:27.572337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:27.572337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:27.591000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:53:27.592328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:27.599488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1146/2301] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:53:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:53:27.611900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:27.624091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef771-5f23-427e-a012-edb5032c84fc service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:27.629339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:27.630260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:27.630604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:27.630883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:27.640544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:27.643376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:27.653315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1156/2302] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:53:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:53:37.585002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=15 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:53:37.648766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=41 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:53:47.295331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:47.304897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93cca-2f6c-4f2e-8bbf-c0c8b5ce924f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:47.309723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:47.310352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:47.310629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:47.310902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:47.319576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:47.321201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:47.330835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1147/2303] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:51.325977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:51.335207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1c01e-a188-4d14-b112-9f66cf5eb9ca service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:51.339050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:51.339839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:51.339959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:51.340211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:51.341983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:51.356040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:51.357586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917a82b-fd31-4086-ba38-505a7222ad5c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:51.373694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1157/2304] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:52.547038 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:52.548605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:52.569261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704f45-b29c-4c9e-a4f0-b8c8898a2d17 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:52.573671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:52.579756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:52.579756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:52.579756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:52.579756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:52.591457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:52.592949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:52.606094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1148/2305] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:53:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:53:56.834806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:56.835290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:56.835407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:56.853671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:53:56.854906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:53:56.861995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1158/2306] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:53:56.874365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:53:56.885991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405b356-2cab-4498-b6e6-c4f980d45ec8 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:53:56.890981 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:53:56.891616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:53:56.891900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:53:56.892176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:53:56.900881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:53:56.902529 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac717-c78a-4024-8b92-0120c51c6adb service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:53:56.910625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1149/2307] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:54:06.139947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:54:06.141462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:06.157218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b71acd-aa29-48e5-85d2-956bbfe6accc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:54:06.162514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:54:06.163144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:06.163422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:06.163690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:06.165412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:06.222927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:54:06.224498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:54:06.234147 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1159/2308] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:54:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:54:06.907392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=18 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:54:15.955185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:15.955868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:15.955868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:15.973700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] MFA Rules not processed for user `f1c9a7f942c541a287da38a894f1d1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:54:15.974947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:15.982324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1150/2309] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:54:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:54:16.230439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=37 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:54:22.314616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:54:22.324360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fdaf4-c028-494e-b25f-7efce53042ea service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:54:22.327863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:54:22.328502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:22.328781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:22.329075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:22.330879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:22.341067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:54:22.342502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:54:22.350622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1160/2310] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:54:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:54:25.968423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:54:32.346947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:54:35.150725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:54:35.163530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb254f0-d48d-4931-a8ac-06c4e504e4bc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:54:35.169543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:54:35.170189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:35.170475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:35.170757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:35.172548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:35.232006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:54:35.233771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:54:35.242899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1151/2311] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:54:45.238432 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=28 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:54:52.861228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:54:52.874274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6fc4-45a2-49ef-82dd-e33fc8c29ddd service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:54:52.879743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:54:52.880376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:52.880667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:52.880945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:52.882807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:52.893188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:54:52.894756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:54:52.903677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1161/2312] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:54:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:54:56.469736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:54:56.481885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4986-3f5f-40e6-94b3-52e797a7da9e service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:54:56.489548 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:54:56.490503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:54:56.490861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:54:56.491193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:54:56.494322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:54:56.513463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:54:56.515196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb8998-523d-4ff7-8d02-038d6ebaa76d service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:54:56.531611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1152/2313] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:54:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:02.899554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:04.135767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:04.166437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35165840-0060-4178-9dfa-951057f3f59d service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:04.172842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:04.173685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:04.174052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:04.174395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:04.176317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:04.221882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:04.223760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:04.245208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1162/2314] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:55:06.523069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:08.632204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:08.632812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:08.632909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:08.650427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] MFA Rules not processed for user `628e662a3b49438f9930a2d160658b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:08.651636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:08.658603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1153/2315] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:08.962058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:08.980958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb0e8-6d84-443c-895b-66ccf52b03cc service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:08.984663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:08.985328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:08.985608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:08.985878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:08.987628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:09.009425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:09.010851 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:09.019428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1163/2316] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:09.416866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:09.418540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:09.494446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052fb7-e3de-4628-8061-e552c84f4ecd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:09.499635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:09.500830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:09.501153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:09.501434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:09.503222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:09.514483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:09.516781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:09.526927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1154/2317] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:15.255502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:15.273003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97cc30-2681-4e5b-aa79-deab2cf5d278 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:15.278767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:15.279415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:15.279698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:15.279974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:15.281772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:15.340694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:15.342318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcc479-8916-4086-8a32-8d6eeb1fa22b service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:15.351856 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1164/2318] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:18.260020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:18.273365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746b882-67ca-4c17-a88f-eff769881eaf service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:18.278958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:18.280928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:18.280928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:18.280928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:18.281917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:18.294217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:18.295912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:18.309052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1155/2319] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:25.347765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=71 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:26.638162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:26.638745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:26.638745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:26.656267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] MFA Rules not processed for user `3e12d902f94b40cea088dc6f24dcb444`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:26.657495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:26.664336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1165/2320] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:28.303389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=41 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:36.651542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:51.934732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:51.943846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408eaa79-48b1-44f6-9eb5-fd3e2b76f391 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:51.948091 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:51.948725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:51.949003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:51.949308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:51.951026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:52.034783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:52.036204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:52.056115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1156/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:55.593936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:55.607273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b1fc-6ae6-441c-879d-e8fd3da95536 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:55.612367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:55.613138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:55.614895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:55.614895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:55.615425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:55.627348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:55.628927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:55.639869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1166/2322] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:55:55.728364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:55.730135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:55.743925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab19e-9e66-4ab5-8b91-a09071b81841 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:55.753117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:55.754014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:55.754299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:55.754577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] PATH_INFO: `/v3/users/f1c9a7f942c541a287da38a894f1d1f9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:55.755797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c9a7f942c541a287da38a894f1d1f9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:55.758308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:55.951023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c9a7f942c541a287da38a894f1d1f9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:55.963316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c9a7f942c541a287da38a894f1d1f9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:55.963972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c9a7f942c541a287da38a894f1d1f9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:55.964579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c9a7f942c541a287da38a894f1d1f9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:55.971401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-130cd751-2765-4b25-a130-6d0f7a8bf8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c9a7f942c541a287da38a894f1d1f9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.071377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1157/2323] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:55:55 2022] DELETE /identity/v3/users/f1c9a7f942c541a287da38a894f1d1f9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:55:56.330315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:56.343118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a784b98-c078-4215-aa09-9fa8bf0fdd62 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:56.420874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:56.421826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:56.422113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:56.422459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] PATH_INFO: `/v3/projects/6e8a673b65b64fbc916da4ce4cdc9f81` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:56.423675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8a673b65b64fbc916da4ce4cdc9f81)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:56.425701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:56.525537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8a673b65b64fbc916da4ce4cdc9f81', 'request_id': 'req-dedd7514-41e4-43e8-b25e-b76dc86567ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.532394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8a673b65b64fbc916da4ce4cdc9f81', 'request_id': 'req-dedd7514-41e4-43e8-b25e-b76dc86567ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.618184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8a673b65b64fbc916da4ce4cdc9f81', 'request_id': 'req-dedd7514-41e4-43e8-b25e-b76dc86567ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.630560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] The token cache is being invalidate because project 6e8a673b65b64fbc916da4ce4cdc9f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:55:56.630854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.631519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dedd7514-41e4-43e8-b25e-b76dc86567ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:56.633209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1167/2324] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:55:56 2022] DELETE /identity/v3/projects/6e8a673b65b64fbc916da4ce4cdc9f81 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:55:56.638915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:56.639209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:56.639547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:56.696452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:56.699818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:56.701074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:56.763346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1158/2325] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:56.771811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:56.772100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:56.772377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:56.794104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:56.795502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:56.803845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1168/2326] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:56.809022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:56.818637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f294d4-18e8-427e-a2fb-5da9ba00f318 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:56.824085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:56.824752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:56.825061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:56.825346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:56.826082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:55:56.826397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:56.828014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:56.839899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1159/2327] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:55:56.844299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:56.853747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b180725-914e-4248-84f0-8f8308d33d83 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:56.858418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:56.859068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:56.859356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:56.859634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:56.860351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:56.862298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:56.957912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1169/2328] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:55:56 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:55:56.958674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:56.969119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d641a1-6a4f-41a5-a232-eb4c921035bc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.027700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.028373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.028660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.028940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.029681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.031662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.107307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1160/2329] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:55:56 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:55:57.108618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.119875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b5e70-534a-4d42-af65-6d0ad9582c85 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.124892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.129904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.129904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.129904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.129904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.129904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.139182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1170/2330] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:55:57.143496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.153797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca01cfc-ea26-4fff-bfd9-fdbfae36a3b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.158460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.159124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.159409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.159689 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.160419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.162067 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889e2f4-a2e0-43de-af7c-2e7b911d9583 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.174822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1161/2331] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:55:57.179710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.190575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aef8d-c5e3-493b-80a2-0ac3bf7f1c50 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.195202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.196223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.196394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.196670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] PATH_INFO: `/v3/projects/d6b3d871743d41a0aa2bd6c27aa09ca4/users/502fc725a7c346b9be36c36db276365d/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.213766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b3d871743d41a0aa2bd6c27aa09ca4, user_id=502fc725a7c346b9be36c36db276365d, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.216415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.276870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1171/2332] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:55:57 2022] PUT /identity/v3/projects/d6b3d871743d41a0aa2bd6c27aa09ca4/users/502fc725a7c346b9be36c36db276365d/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:55:57.286895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.287180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.287459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.315401 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] MFA Rules not processed for user `502fc725a7c346b9be36c36db276365d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:57.316604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:57.380438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1162/2333] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:57.385274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.390999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:57.446397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7a9b8-2986-405e-93f8-c4dea217d4ce admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.450991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.451646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.451930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.452206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.452908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.455326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.570037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1172/2334] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:55:57 2022] POST /identity/v3/projects => generated 371 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:55:57.574411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.579904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:57.585588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9d1cd-c829-48e2-bbc4-286f09acbde5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.642319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.642903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.643184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.643460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.644161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.646174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.711076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1163/2335] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:55:57 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:55:57.715640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.726451 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f195-dffe-4265-a362-6cb225477546 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.732111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.732772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.733093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.733375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.734105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.735727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.746848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1173/2336] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:55:57.750937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.761302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349be0a-cffd-443e-877c-c8ce8c3f0b32 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.765916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.766793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.767075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.767355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] PATH_INFO: `/v3/projects/140cbba0457b48f285dc2221f983aa55/users/b0286d22844d4678a45471de5861e63f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.782692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140cbba0457b48f285dc2221f983aa55, user_id=b0286d22844d4678a45471de5861e63f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.785371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.872223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1164/2337] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:55:57 2022] PUT /identity/v3/projects/140cbba0457b48f285dc2221f983aa55/users/b0286d22844d4678a45471de5861e63f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:55:57.876160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.887577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e66b8-e037-4754-8f10-0cdb1261f4d0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.945311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.946662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.947176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.947453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.948202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.949817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe9560-0626-4e81-8a97-df19b43b886b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.959415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1174/2338] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:55:57.964237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:57.974702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b4904-c6e4-4243-ad2b-22c6259cfb84 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:57.979293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:57.985580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:57.985580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:57.985580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] PATH_INFO: `/v3/projects/140cbba0457b48f285dc2221f983aa55/users/b0286d22844d4678a45471de5861e63f/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:57.985580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140cbba0457b48f285dc2221f983aa55, user_id=b0286d22844d4678a45471de5861e63f, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:57.985580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:57.995605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1165/2339] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:55:57 2022] PUT /identity/v3/projects/140cbba0457b48f285dc2221f983aa55/users/b0286d22844d4678a45471de5861e63f/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:55:58.006040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:58.006354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:58.006635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:58.033432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] MFA Rules not processed for user `b0286d22844d4678a45471de5861e63f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:58.034646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:58.101436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:58.105026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1175/2340] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:58.116534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:58.118337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:58.199816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb7c-183b-47be-aa07-3ede19de40b2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:58.203608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:58.204364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:58.204645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:58.204918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:58.214732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:58.216184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ea8c7-efa1-4512-b3b0-7e5d2b0eb99d service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:58.225680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1166/2341] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:55:59.069160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:59.134721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e72d32-3870-4f28-83e6-09f9a5d327a1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:59.139336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:59.140921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:59.140921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:59.140921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] PATH_INFO: `/v3/users/502fc725a7c346b9be36c36db276365d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:59.142051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502fc725a7c346b9be36c36db276365d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:59.143979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:59.405609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502fc725a7c346b9be36c36db276365d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:59.443743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502fc725a7c346b9be36c36db276365d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:59.463536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502fc725a7c346b9be36c36db276365d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:59.464608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502fc725a7c346b9be36c36db276365d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:59.574394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502fc725a7c346b9be36c36db276365d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:55:59.581572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1176/2342] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:55:59 2022] DELETE /identity/v3/users/502fc725a7c346b9be36c36db276365d => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:55:59.587451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:59.587996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:59.588110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:59.619245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:55:59.620609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:59.672923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:55:59.701475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1167/2343] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:55:59.713088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:55:59.714547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:55:59.788106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f75e-45bf-45db-a846-12c435ada5d7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:55:59.792978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:55:59.793828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:55:59.794183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:55:59.794904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:55:59.805838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:55:59.807684 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:55:59.820523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1177/2344] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:00.434294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:00.435864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:00.458200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e817e-b121-4597-94e2-0f6bf0864d71 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:00.474198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:00.474198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:00.474198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:00.474198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] PATH_INFO: `/v3/projects/d6b3d871743d41a0aa2bd6c27aa09ca4` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:00.474530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b3d871743d41a0aa2bd6c27aa09ca4)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:00.474530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:00.580382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b3d871743d41a0aa2bd6c27aa09ca4', 'request_id': 'req-be4556c0-2a34-4ded-9318-ebc96941f9c3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.587054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b3d871743d41a0aa2bd6c27aa09ca4', 'request_id': 'req-be4556c0-2a34-4ded-9318-ebc96941f9c3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.688430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b3d871743d41a0aa2bd6c27aa09ca4', 'request_id': 'req-be4556c0-2a34-4ded-9318-ebc96941f9c3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.699838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] The token cache is being invalidate because project d6b3d871743d41a0aa2bd6c27aa09ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:00.700673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.701360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.705904 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1168/2345] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:00 2022] DELETE /identity/v3/projects/d6b3d871743d41a0aa2bd6c27aa09ca4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:00.707309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:00.717905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160144d0-bdf1-4880-9284-f05b720a580a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:00.738762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:00.778119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:00.779007 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:00.779287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:00.779559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] PATH_INFO: `/v3/users/b0286d22844d4678a45471de5861e63f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:00.780923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0286d22844d4678a45471de5861e63f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:00.782960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:00.983442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0286d22844d4678a45471de5861e63f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:00.990436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0286d22844d4678a45471de5861e63f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.003656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0286d22844d4678a45471de5861e63f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.004396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0286d22844d4678a45471de5861e63f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.059726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12681abb-8a16-4d47-8da8-3097086e1ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0286d22844d4678a45471de5861e63f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.066784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1178/2346] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:00 2022] DELETE /identity/v3/users/b0286d22844d4678a45471de5861e63f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:01.502990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:01.504725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:01.528061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4556c0-2a34-4ded-9318-ebc96941f9c3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:01.592910 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:01.593820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:01.594103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:01.594379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] PATH_INFO: `/v3/projects/140cbba0457b48f285dc2221f983aa55` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:01.595577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=140cbba0457b48f285dc2221f983aa55)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:01.597521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:01.700549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140cbba0457b48f285dc2221f983aa55', 'request_id': 'req-7d836114-07d1-4a70-8226-f61f9bbd2e6a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.707041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140cbba0457b48f285dc2221f983aa55', 'request_id': 'req-7d836114-07d1-4a70-8226-f61f9bbd2e6a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.782927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140cbba0457b48f285dc2221f983aa55', 'request_id': 'req-7d836114-07d1-4a70-8226-f61f9bbd2e6a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.785493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:01.795185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] The token cache is being invalidate because project 140cbba0457b48f285dc2221f983aa55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:01.795470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.796112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7d836114-07d1-4a70-8226-f61f9bbd2e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:01.798716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1169/2347] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:01 2022] DELETE /identity/v3/projects/140cbba0457b48f285dc2221f983aa55 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:01.802867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:01.803322 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:01.803416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:01.824976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:01.826804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:01.890784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1179/2348] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:01.900458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:01.900743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:01.901020 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:01.921501 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:01.922740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:01.932728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1170/2349] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:01.936825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:01.952439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385b32a-604c-4f0a-9785-fd11ffc0dfc3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:01.958183 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:01.958853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:01.959134 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:01.959469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:01.960365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:01.960686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:01.963138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:01.984954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1180/2350] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:01.986148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:01.996092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb81d98-e8bb-4bc2-9062-2d91b4eda5f2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.000863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.001777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.002235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.002519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.003217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.005320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.130666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1171/2351] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:01 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:02.135764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.146718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b44c-47f0-4f37-8e9f-b6089a4d4e0a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.186100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:02.225707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.225707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.225707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.225707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.226115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.226926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.309356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1181/2352] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:02 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:02.310619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.321606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75c7e-e965-41f1-8156-99ac247abdab admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.333048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.333945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.334211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.334503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.335239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.337831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.353007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1172/2353] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:02.356629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.376259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb530e3-23df-46b8-bfb6-250bdbcb2c5e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.381644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.385998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.385998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.385998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.385998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.385998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07725695-24fb-4a60-af32-2aa9b3569b21 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.397099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1182/2354] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:02.402366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.412740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07207e-8bf0-464b-b95c-8069813afd37 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.417817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.418702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.418984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.419254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] PATH_INFO: `/v3/projects/ae9cc75bfdc043008d038ce68821c98f/users/3e88a2032b524235bb6269c8049cff1e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.434555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9cc75bfdc043008d038ce68821c98f, user_id=3e88a2032b524235bb6269c8049cff1e, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.437243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.540687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1173/2355] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:02 2022] PUT /identity/v3/projects/ae9cc75bfdc043008d038ce68821c98f/users/3e88a2032b524235bb6269c8049cff1e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:02.550968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.551265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.551544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.580870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] MFA Rules not processed for user `3e88a2032b524235bb6269c8049cff1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:02.582224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:02.647528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1183/2356] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:02.657326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.718494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776d86a-bd51-4282-bede-8457a141ccbe admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.723229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.723916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.724197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.724467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.725179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.727699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.801587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1174/2357] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:02 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:02.806565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.817614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a03d1-a73e-4aca-9bc9-1e5de78d479f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:02.874244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:02.874901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:02.875185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:02.875465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:02.876142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:02.878064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:02.985526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1184/2358] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:02 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:02.989735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:02.999932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487f99c-3df6-4ae9-ae13-58340f90f995 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:03.004569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:03.005251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.005535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.005812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.006534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:03.008120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:03.018579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1175/2359] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:03.022641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:03.033458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b4f5-648a-4965-94e9-ab6e46271bad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:03.038270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:03.039224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.040252 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.040769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] PATH_INFO: `/v3/projects/e190415eda114880baf4aca2dbcc7b89/users/3084166506a341e7ad9b6810030682d5/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.055746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e190415eda114880baf4aca2dbcc7b89, user_id=3084166506a341e7ad9b6810030682d5, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:03.058401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:03.203718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1185/2360] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:03 2022] PUT /identity/v3/projects/e190415eda114880baf4aca2dbcc7b89/users/3084166506a341e7ad9b6810030682d5/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:03.207764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:03.218554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b244-374e-4913-ac78-e1e2340f7ad8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:03.272971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:03.273655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.274512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.274866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.275586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:03.277204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfe9c-63b6-4a2c-a8de-0da39332984e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:03.279201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:03.287146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1176/2361] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:03.291575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:03.297480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:03.302727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b36f2-344f-497a-a8d2-e30fb95883b2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:03.308155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:03.308972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.309287 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.309565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] PATH_INFO: `/v3/projects/e190415eda114880baf4aca2dbcc7b89/users/3084166506a341e7ad9b6810030682d5/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.311519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e190415eda114880baf4aca2dbcc7b89, user_id=3084166506a341e7ad9b6810030682d5, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:03.314216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:03.325059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1186/2362] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:03 2022] PUT /identity/v3/projects/e190415eda114880baf4aca2dbcc7b89/users/3084166506a341e7ad9b6810030682d5/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:03.335216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.335503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.335778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.362095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] MFA Rules not processed for user `3084166506a341e7ad9b6810030682d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:03.363293 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:03.430559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1177/2363] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:03.441740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:03.444491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:03.503838 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db7b7b-614a-4d1a-892a-e29a233857d3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:03.507896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:03.509558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:03.509760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:03.510113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:03.519726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:03.521238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fab84-5256-4e8a-87c5-9096df968c43 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:03.531052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1187/2364] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:06.344495 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:06.414724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29d1af-0da2-4df2-87e8-ab47846315bc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:06.419916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:06.420796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:06.421133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:06.421420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] PATH_INFO: `/v3/users/3e88a2032b524235bb6269c8049cff1e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:06.422698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e88a2032b524235bb6269c8049cff1e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:06.424992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:06.605428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e88a2032b524235bb6269c8049cff1e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:06.619328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e88a2032b524235bb6269c8049cff1e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:06.620712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e88a2032b524235bb6269c8049cff1e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:06.621387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e88a2032b524235bb6269c8049cff1e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:06.628574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e88a2032b524235bb6269c8049cff1e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:06.687077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1178/2365] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:06 2022] DELETE /identity/v3/users/3e88a2032b524235bb6269c8049cff1e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:06.691640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:06.692117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:06.692211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:06.710493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:06.711680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:06.770921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1188/2366] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:06.780609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:06.782051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:06.787968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:06.843589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed91c7-8931-4d28-918e-17ad73689ad6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:06.847242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:06.847883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:06.848166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:06.848491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:06.858015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:06.859482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:06.868898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1179/2367] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:07.284752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:07.286198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:07.298701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fb7aa-93ab-4403-8316-c5b52013f657 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:07.303228 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:07.304077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:07.304605 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:07.304878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] PATH_INFO: `/v3/projects/ae9cc75bfdc043008d038ce68821c98f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:07.306196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9cc75bfdc043008d038ce68821c98f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:07.308638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:07.409277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9cc75bfdc043008d038ce68821c98f', 'request_id': 'req-d3458d16-992c-42fe-902d-3e2155b41147'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.415376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9cc75bfdc043008d038ce68821c98f', 'request_id': 'req-d3458d16-992c-42fe-902d-3e2155b41147'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.476566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9cc75bfdc043008d038ce68821c98f', 'request_id': 'req-d3458d16-992c-42fe-902d-3e2155b41147'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.488388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] The token cache is being invalidate because project ae9cc75bfdc043008d038ce68821c98f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:07.488748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.489395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.490932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1189/2368] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:07 2022] DELETE /identity/v3/projects/ae9cc75bfdc043008d038ce68821c98f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:07.495182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:07.508035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fda22-27be-4d76-b7be-d873068ad8b4 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:07.568547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:07.569405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:07.569685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:07.570082 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] PATH_INFO: `/v3/users/3084166506a341e7ad9b6810030682d5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:07.571279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3084166506a341e7ad9b6810030682d5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:07.573768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:07.742805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3084166506a341e7ad9b6810030682d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.757534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3084166506a341e7ad9b6810030682d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.761117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3084166506a341e7ad9b6810030682d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.761117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3084166506a341e7ad9b6810030682d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.766270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5ddcba6d-bd1c-4480-a364-30f92124d312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3084166506a341e7ad9b6810030682d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:07.849430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1180/2369] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:07 2022] DELETE /identity/v3/users/3084166506a341e7ad9b6810030682d5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:08.400093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:08.403254 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:08.417468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3458d16-992c-42fe-902d-3e2155b41147 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:08.435269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:08.537064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:08.538001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:08.538289 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:08.538559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] PATH_INFO: `/v3/projects/e190415eda114880baf4aca2dbcc7b89` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:08.539826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e190415eda114880baf4aca2dbcc7b89)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:08.542049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:08.651847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e190415eda114880baf4aca2dbcc7b89', 'request_id': 'req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:08.658996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e190415eda114880baf4aca2dbcc7b89', 'request_id': 'req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:08.762960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e190415eda114880baf4aca2dbcc7b89', 'request_id': 'req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:08.776454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] The token cache is being invalidate because project e190415eda114880baf4aca2dbcc7b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:08.776720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:08.777403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5d86bd1c-6e35-470c-b1d2-6993cc22e5c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:08.779235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1190/2370] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:08 2022] DELETE /identity/v3/projects/e190415eda114880baf4aca2dbcc7b89 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:08.784014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:08.784521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:08.784633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:08.806376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:08.808324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:08.837614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:08.886682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1181/2371] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:08.897143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:08.897143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:08.897143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:08.918512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:08.919987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:08.928770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1191/2372] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:08.933649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:08.948958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65d1ed-ba93-4101-aa8f-1071af38e63a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:08.954859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:08.955619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:08.955900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:08.956171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:08.956999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:08.957243 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:08.958868 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:08.975742 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1182/2373] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:08.977266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:08.987998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018a5d0-1fcf-4f5b-a46c-016c2c54a01d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:08.996417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:08.997292 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.000547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.000547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.000547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.000547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.101972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1192/2374] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:08 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:09.106212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.116173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657d097-318c-4bc3-8966-168a01643129 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.172037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.172636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.172923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.173229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.173937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.176656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.253298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1183/2375] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:09 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:09.257273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.268910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe89f-03b7-463e-85de-1b529a9fd210 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.273622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.274276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.274562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.274833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.275552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.278095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.288904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1193/2376] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:09.293646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.304917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f35079-682f-4bc1-93d8-48d661e64486 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.310058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.310726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.311002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.311283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.312048 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.313775 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60fe5e-71dd-4b31-825a-d2b2269f483e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.324969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1184/2377] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:09.329601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.341223 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad0fb-2b26-4cf2-b430-e986605a428f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.346238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.347143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.347429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.347704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] PATH_INFO: `/v3/projects/32209507688e4bf09fc6e754b3caf4a1/users/0e1194b4eac44bc18e60fbb7c79b3b8c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.363157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32209507688e4bf09fc6e754b3caf4a1, user_id=0e1194b4eac44bc18e60fbb7c79b3b8c, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.365938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.455876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1194/2378] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:09 2022] PUT /identity/v3/projects/32209507688e4bf09fc6e754b3caf4a1/users/0e1194b4eac44bc18e60fbb7c79b3b8c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:09.466614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.466909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.467197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.497574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] MFA Rules not processed for user `0e1194b4eac44bc18e60fbb7c79b3b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:09.498063 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:09.588219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1185/2379] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:09.592844 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.655826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:09.675260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1ddc-67da-4ead-8b21-3106ac8a4d33 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.681230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.681561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.681849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.682126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.683963 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.686325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aabc18-f274-405e-9941-b747e841c9b5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.773700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1195/2380] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:09 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:09.778434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:09.793466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78c109-4140-4303-bcdd-a380968df10d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:09.818620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:09.868486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:09.869391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.870436 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.870638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.871349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:09.873455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:09.949690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:09.950041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:09.950360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:09.981274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:09.982544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:09.989361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1186/2381] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:09 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:09.995889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.008940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999118b-179e-4988-9cde-d732ec97caba admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.014284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.014973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.015415 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.015926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.017094 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.019390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.036894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1187/2382] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:10.038169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.051818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56862e-0462-4dba-88f6-8a3c8684a6ac admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.053711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1196/2383] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 12:56:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:10.058192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.059527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.059819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.060098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] PATH_INFO: `/v3/projects/903c42c4076946d0bbaf85135fb293c0/users/f71d590b92c14d059f85fc503b77cc6c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.095685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903c42c4076946d0bbaf85135fb293c0, user_id=f71d590b92c14d059f85fc503b77cc6c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.101993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.208198 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1188/2384] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:10 2022] PUT /identity/v3/projects/903c42c4076946d0bbaf85135fb293c0/users/f71d590b92c14d059f85fc503b77cc6c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:10.213950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.230008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4c112-9ede-4e9e-83ac-0b4a99b9bf8d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.237451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.239562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:10.314755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.315497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.315895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.316105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.316831 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.319120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.329127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa4678-f443-47ec-aa78-84af039050c7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.337123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1197/2385] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:10.340472 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.344936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.345676 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.346003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.346283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.356752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc2299-e759-4d8a-91b3-23de2b1f04d2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.362947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.363988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.364346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.364650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] PATH_INFO: `/v3/projects/903c42c4076946d0bbaf85135fb293c0/users/f71d590b92c14d059f85fc503b77cc6c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.366983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903c42c4076946d0bbaf85135fb293c0, user_id=f71d590b92c14d059f85fc503b77cc6c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.370435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7bbfd-1da4-4f32-a39b-6cafee8fe801 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.387266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1198/2386] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:10 2022] PUT /identity/v3/projects/903c42c4076946d0bbaf85135fb293c0/users/f71d590b92c14d059f85fc503b77cc6c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:10.399119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.399410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.399690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.435650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] MFA Rules not processed for user `f71d590b92c14d059f85fc503b77cc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:10.437357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:10.439104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.441648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.455999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1189/2387] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:56:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:56:10.485824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:10.533185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1199/2388] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:10 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:10.546201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.547721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:10.579163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:10.618949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ec702-ff1f-4da1-beca-8fb8d76bdec5 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.669818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.673522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.673921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.675044 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.686707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.688822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcd8ce-5d1f-4099-8ecc-69b0c7034963 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.699367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1190/2389] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:10 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:10.781561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:10.783371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:10.796989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b127-ffd2-4fa7-a15d-dc2c4fd2c287 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:10.800628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:10.801578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:10.801854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:10.802203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:10.864528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:10.866004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:10.876071 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1200/2390] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:15.153706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e408-cdf0-4112-8de8-859696f0d88c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:15.154241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e408-cdf0-4112-8de8-859696f0d88c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:15.154241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e408-cdf0-4112-8de8-859696f0d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:15.182725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-02d1e408-cdf0-4112-8de8-859696f0d88c None None] MFA Rules not processed for user `46b5c80d8bf34965a30b9b67e9117996`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:15.184785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-02d1e408-cdf0-4112-8de8-859696f0d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:15.256267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1191/2391] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:15.971755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:16.050336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2bd6-cad2-4519-81e8-d4d66d3f2d1f service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:16.058200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:16.059238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:16.059526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:16.059798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] PATH_INFO: `/v3/users/0e1194b4eac44bc18e60fbb7c79b3b8c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:16.061640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1194b4eac44bc18e60fbb7c79b3b8c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:16.063283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:16.296859 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1194b4eac44bc18e60fbb7c79b3b8c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:16.305325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1194b4eac44bc18e60fbb7c79b3b8c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:16.318771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:16.320265 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1194b4eac44bc18e60fbb7c79b3b8c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:16.321080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1194b4eac44bc18e60fbb7c79b3b8c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:16.414869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1194b4eac44bc18e60fbb7c79b3b8c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:16.424945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1201/2392] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:15 2022] DELETE /identity/v3/users/0e1194b4eac44bc18e60fbb7c79b3b8c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:16.431391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:16.431803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:16.431962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:16.476608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:16.478332 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:16.572242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1192/2393] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:16.583826 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:16.585358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:16.650533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f9b0-3e3e-406d-abac-bc08f135a801 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:16.651116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:16.651835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:16.652271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:16.652692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:16.662817 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:16.664402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:16.676629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1202/2394] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:17.159553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:17.161679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:17.177946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad41e98-bc8e-46e2-af14-d7c4d43c5f62 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:17.188588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:17.189629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:17.189969 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:17.190307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] PATH_INFO: `/v3/projects/32209507688e4bf09fc6e754b3caf4a1` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:17.191787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32209507688e4bf09fc6e754b3caf4a1)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:17.194195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:17.213235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:17.326768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32209507688e4bf09fc6e754b3caf4a1', 'request_id': 'req-dfe4446a-e807-48f1-bd1c-499fcde2a865'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.334790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32209507688e4bf09fc6e754b3caf4a1', 'request_id': 'req-dfe4446a-e807-48f1-bd1c-499fcde2a865'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.437620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32209507688e4bf09fc6e754b3caf4a1', 'request_id': 'req-dfe4446a-e807-48f1-bd1c-499fcde2a865'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.458376 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] The token cache is being invalidate because project 32209507688e4bf09fc6e754b3caf4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:17.459538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.461725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.466682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1193/2395] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:17 2022] DELETE /identity/v3/projects/32209507688e4bf09fc6e754b3caf4a1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:17.472074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:17.483604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb9d67-8d99-4777-b79a-2f207445135c service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:17.558762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:17.559672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:17.560049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:17.560331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] PATH_INFO: `/v3/users/f71d590b92c14d059f85fc503b77cc6c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:17.561575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71d590b92c14d059f85fc503b77cc6c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:17.563524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:17.760350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71d590b92c14d059f85fc503b77cc6c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.768469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71d590b92c14d059f85fc503b77cc6c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.782234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71d590b92c14d059f85fc503b77cc6c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.783176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71d590b92c14d059f85fc503b77cc6c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.863376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09838d19-bc61-4341-bc8b-0f7631f0909d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71d590b92c14d059f85fc503b77cc6c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:17.871767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1203/2396] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:17 2022] DELETE /identity/v3/users/f71d590b92c14d059f85fc503b77cc6c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:19.103351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:19.106533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:19.140896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4446a-e807-48f1-bd1c-499fcde2a865 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:19.210901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:19.212261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.212391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.212653 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] PATH_INFO: `/v3/projects/903c42c4076946d0bbaf85135fb293c0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.214914 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903c42c4076946d0bbaf85135fb293c0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:19.217337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:19.362670 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903c42c4076946d0bbaf85135fb293c0', 'request_id': 'req-1427e5db-837a-4e7b-a145-df7bc85d97b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:19.368666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903c42c4076946d0bbaf85135fb293c0', 'request_id': 'req-1427e5db-837a-4e7b-a145-df7bc85d97b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:19.494562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903c42c4076946d0bbaf85135fb293c0', 'request_id': 'req-1427e5db-837a-4e7b-a145-df7bc85d97b7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:19.510396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] The token cache is being invalidate because project 903c42c4076946d0bbaf85135fb293c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:19.511213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:19.512132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1427e5db-837a-4e7b-a145-df7bc85d97b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:19.514368 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1194/2397] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:19 2022] DELETE /identity/v3/projects/903c42c4076946d0bbaf85135fb293c0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:19.519315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.519847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.520001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.542940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:19.544401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:19.554396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:19.639632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1204/2398] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:19.647312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.647682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.654142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.667743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:19.668934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:19.681596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1195/2399] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:19.684757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:19.700236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69dc2-900e-40db-ad14-0db935365566 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:19.707500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:19.708486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.708968 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.709454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.710541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:19.711013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:19.713279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:19.730916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1205/2400] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:19.735026 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:19.758931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1640fa-1da3-4cec-9bcd-34e557c0fe63 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:19.765352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:19.765871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.766257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.766642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.767803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:19.770439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:19.866269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1196/2401] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:19 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:19.867566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:19.883453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcaf2-1102-494e-b46e-8b162d57165b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:19.960403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:19.961758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:19.962143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:19.962548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:19.963683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:19.967359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.097217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1206/2402] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:19 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:20.098693 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.106847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:20.115661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2eb6-1667-4f9a-9971-63a5856e46d3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.118947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.119673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.119960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.120897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.122163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.123761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.134781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1197/2403] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:20.139235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.150080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4801433-fb8f-444b-96ff-f1e2d1b00dd0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.154845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.155508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.155792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.156068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.156798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.158456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.169758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1207/2404] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:20.177750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.189667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44136-e939-47bc-9e47-cc1a6b83d532 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.194524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.195461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.195741 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.196015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] PATH_INFO: `/v3/projects/7660b8f3660d47ecab0fa40bbdbb67b6/users/8287f16ac7214990ae6870d57580f1cc/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.210779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7660b8f3660d47ecab0fa40bbdbb67b6, user_id=8287f16ac7214990ae6870d57580f1cc, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.213535 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27a1de-b523-4ae2-8097-85717ee168a2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.242692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.244646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:20.262242 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abee4-c442-4668-bc62-319e0fae3da1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.266984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.272906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.272906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.272906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] PATH_INFO: `/v3/users/8171d4ef9f644866bb5747a6c31fa5cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.272906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8171d4ef9f644866bb5747a6c31fa5cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.272906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.320715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1198/2405] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:20 2022] PUT /identity/v3/projects/7660b8f3660d47ecab0fa40bbdbb67b6/users/8287f16ac7214990ae6870d57580f1cc/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:20.331254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.331543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.331818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.361298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] MFA Rules not processed for user `8287f16ac7214990ae6870d57580f1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:20.362628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:20.429207 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1199/2406] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:20.434076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.509084 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b1cf5-f480-47f6-8a82-67160d085e16 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.514346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.515283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.515757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.516219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.517150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.518456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8171d4ef9f644866bb5747a6c31fa5cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:20.519566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.527554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8171d4ef9f644866bb5747a6c31fa5cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:20.540643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:20.544483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8171d4ef9f644866bb5747a6c31fa5cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:20.546521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8171d4ef9f644866bb5747a6c31fa5cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:20.613190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1200/2407] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:20 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:20.615338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.630162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8171d4ef9f644866bb5747a6c31fa5cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:20.637617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1208/2408] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:20 2022] DELETE /identity/v3/users/8171d4ef9f644866bb5747a6c31fa5cf => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:20.717950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a13-c07c-4d2a-ab91-6c0a5cd2cccd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.753117 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:20.816616 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.817425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.817712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.817989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.819100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.821108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:20.948714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1201/2409] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:20 2022] POST /identity/v3/users => generated 296 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:20.949946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:20.952138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:20.973338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b5fb4-64ad-4dc5-aff8-bbbf3902be5d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:20.982755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:20.983513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:20.983825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:20.984102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:20.984837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:20.987258 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.004448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1209/2410] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:21.005940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.022123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebfcd-4516-40fa-bb3b-67cf97f53323 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.027249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.029354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.029354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.029354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] PATH_INFO: `/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd/users/7f33c1904366498aae19b6333f0480cc/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.047540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c0e6f986ed4e91a5854fa4e26da6cd, user_id=7f33c1904366498aae19b6333f0480cc, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.050252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.120718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.122646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:21.138015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac589b-5382-4fc1-98cf-65ed51e68463 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.145540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1202/2411] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:21 2022] PUT /identity/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd/users/7f33c1904366498aae19b6333f0480cc/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:21.149329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.217243 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.218221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.218570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.218966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] PATH_INFO: `/v3/projects/fa16115e6ada4f16874b4f48730bec7f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.220428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa16115e6ada4f16874b4f48730bec7f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.222505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.224307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37998976-a409-4ff9-bcf1-2e592fab4c72 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.228272 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.228953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.229274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.229918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.230608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.232337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.243092 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1203/2412] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:56:21.247655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.258730 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992bef-8933-4e8f-a9ec-2cca3efd928a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.264559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.265599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.265881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.266158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] PATH_INFO: `/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd/users/7f33c1904366498aae19b6333f0480cc/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.268152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c0e6f986ed4e91a5854fa4e26da6cd, user_id=7f33c1904366498aae19b6333f0480cc, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.270984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd3ba6-372a-469c-af9d-a6c14df9c236 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.283458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1204/2413] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:21 2022] PUT /identity/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd/users/7f33c1904366498aae19b6333f0480cc/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:21.294909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.295219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.295525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.324242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] MFA Rules not processed for user `7f33c1904366498aae19b6333f0480cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:21.325656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:21.347431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa16115e6ada4f16874b4f48730bec7f', 'request_id': 'req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.353842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa16115e6ada4f16874b4f48730bec7f', 'request_id': 'req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.357271 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:21.398791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1205/2414] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:21.411410 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.412774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:21.425885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa16115e6ada4f16874b4f48730bec7f', 'request_id': 'req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.441625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] The token cache is being invalidate because project fa16115e6ada4f16874b4f48730bec7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:21.446488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.446488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.446488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1210/2415] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:21 2022] DELETE /identity/v3/projects/fa16115e6ada4f16874b4f48730bec7f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:21.449642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:21.451127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:21.478216 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699ed64-03f7-46b3-90f5-56e7ba305eb8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.524909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cc8cc-2f3a-4ad0-abdf-31c8b486d913 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:21.530106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.531065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.531658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.532120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] PATH_INFO: `/v3/users/ee4ca294ed084d12bceed71bef4d1615` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.533364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:21.533517 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4ca294ed084d12bceed71bef4d1615)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.533966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:21.535143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:21.535143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:21.535456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.537917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:21.548755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:21.609414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:21.610851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:21.621752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1206/2416] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:56:21.737013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4ca294ed084d12bceed71bef4d1615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.743528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4ca294ed084d12bceed71bef4d1615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.755321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4ca294ed084d12bceed71bef4d1615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.755982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4ca294ed084d12bceed71bef4d1615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.817750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4ca294ed084d12bceed71bef4d1615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:21.824735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1211/2417] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:21 2022] DELETE /identity/v3/users/ee4ca294ed084d12bceed71bef4d1615 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:22.048295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:22.068561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826edd6-73e1-4289-bc00-d8c38e42c1f7 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:22.103199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:22.133248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:22.134323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:22.134610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:22.134892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] PATH_INFO: `/v3/projects/18a6f76c9d664670b52552f99558398f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:22.136364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a6f76c9d664670b52552f99558398f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:22.138309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:22.240307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a6f76c9d664670b52552f99558398f', 'request_id': 'req-262ccc17-b315-47f2-8f85-407fbed49365'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:22.246526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a6f76c9d664670b52552f99558398f', 'request_id': 'req-262ccc17-b315-47f2-8f85-407fbed49365'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:22.331671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a6f76c9d664670b52552f99558398f', 'request_id': 'req-262ccc17-b315-47f2-8f85-407fbed49365'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:22.345008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] The token cache is being invalidate because project 18a6f76c9d664670b52552f99558398f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:22.345326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:22.345960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:22.347508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1207/2418] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:22 2022] DELETE /identity/v3/projects/18a6f76c9d664670b52552f99558398f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:22.671177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:22.672632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:22.736819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf84b9-152a-4e2d-87ee-ea54ed599899 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:22.791531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f33c1904366498aae19b6333f0480cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64c0e6f986ed4e91a5854fa4e26da6cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:22.792195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:22.792485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:22.792764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:22.793533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:22.794419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:22.796027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:22.807648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1212/2419] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:22.812342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:22.813768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:22.826908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ccc17-b315-47f2-8f85-407fbed49365 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:22.831318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f33c1904366498aae19b6333f0480cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64c0e6f986ed4e91a5854fa4e26da6cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:22.831960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:22.832244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:22.832524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:22.833247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:22.835125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:22.910939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1208/2420] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:22 2022] POST /identity/v3/projects => generated 353 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:22.915050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:22.925321 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792109fc-3b0c-47d3-9048-7bc5b7ef5c21 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:22.958065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:22.988406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f33c1904366498aae19b6333f0480cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64c0e6f986ed4e91a5854fa4e26da6cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:22.988999 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:22.989323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:22.989604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:22.990326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:22.990639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:22.992256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:23.003470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1213/2421] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:23.039973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:23.050417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a95552-efcb-473f-b634-2cb0bc7c7bdd tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:23.055253 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f33c1904366498aae19b6333f0480cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64c0e6f986ed4e91a5854fa4e26da6cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:23.056158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:23.056452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:23.056732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] PATH_INFO: `/v3/projects/5e62a7d386754f1894e97de1538493fa` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:23.057967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5e62a7d386754f1894e97de1538493fa)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:23.060480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:23.166523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e62a7d386754f1894e97de1538493fa', 'request_id': 'req-ec936023-e924-4b8f-b789-17bc3766a5fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:23.173054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e62a7d386754f1894e97de1538493fa', 'request_id': 'req-ec936023-e924-4b8f-b789-17bc3766a5fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:23.269007 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e62a7d386754f1894e97de1538493fa', 'request_id': 'req-ec936023-e924-4b8f-b789-17bc3766a5fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:23.281728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] The token cache is being invalidate because project 5e62a7d386754f1894e97de1538493fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:23.283696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:23.283696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ec936023-e924-4b8f-b789-17bc3766a5fb tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:23.287609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1209/2422] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:23 2022] DELETE /identity/v3/projects/5e62a7d386754f1894e97de1538493fa => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:23.861639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:23.863401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:23.928598 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818ed70-a870-4557-9589-7abec23c9e24 tempest-VolumeQuotaClassesTest-835153730 tempest-VolumeQuotaClassesTest-835153730-project-admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:23.984770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:23.985689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:23.985977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:23.996401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] PATH_INFO: `/v3/users/8287f16ac7214990ae6870d57580f1cc` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:23.996401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8287f16ac7214990ae6870d57580f1cc)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:23.996401 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:24.202303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8287f16ac7214990ae6870d57580f1cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:24.209895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8287f16ac7214990ae6870d57580f1cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:24.221946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:24.223246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8287f16ac7214990ae6870d57580f1cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:24.224060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8287f16ac7214990ae6870d57580f1cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:24.306378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8287f16ac7214990ae6870d57580f1cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:24.313726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1214/2423] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:23 2022] DELETE /identity/v3/users/8287f16ac7214990ae6870d57580f1cc => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:24.318220 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:24.318701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:24.318822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:24.342416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:24.343641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:24.360780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:24.408718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1210/2424] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:24.419536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:24.421230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:24.480169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a32569-0e3b-4391-894b-26e63c9a2c50 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:24.484389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:24.485057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:24.485347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:24.485621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:24.495064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:24.497318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:24.507477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1215/2425] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:24.982104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:24.983731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:24.995715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f1622-34dc-4f7a-adae-1370cae97f6c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:25.000424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:25.001317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:25.001594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:25.001867 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] PATH_INFO: `/v3/projects/7660b8f3660d47ecab0fa40bbdbb67b6` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:25.003036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7660b8f3660d47ecab0fa40bbdbb67b6)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:25.005205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:25.121268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7660b8f3660d47ecab0fa40bbdbb67b6', 'request_id': 'req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.127692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7660b8f3660d47ecab0fa40bbdbb67b6', 'request_id': 'req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.206414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7660b8f3660d47ecab0fa40bbdbb67b6', 'request_id': 'req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.217793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] The token cache is being invalidate because project 7660b8f3660d47ecab0fa40bbdbb67b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:25.218076 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.218688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.220214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1211/2426] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:24 2022] DELETE /identity/v3/projects/7660b8f3660d47ecab0fa40bbdbb67b6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:25.226909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:25.234203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351f088-e164-478b-b666-44ef75cb7da0 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:25.290941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:25.291799 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:25.292076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:25.292351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] PATH_INFO: `/v3/users/7f33c1904366498aae19b6333f0480cc` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:25.293543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f33c1904366498aae19b6333f0480cc)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:25.295465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:25.535984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f33c1904366498aae19b6333f0480cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.542728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f33c1904366498aae19b6333f0480cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.554314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f33c1904366498aae19b6333f0480cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.555232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f33c1904366498aae19b6333f0480cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.642852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ffa99b8-cd2d-40a8-84b9-1db0623ad44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f33c1904366498aae19b6333f0480cc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:25.649371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1216/2427] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:25 2022] DELETE /identity/v3/users/7f33c1904366498aae19b6333f0480cc => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:26.101611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.103069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:26.117903 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3322b9-7fa8-48a3-8a06-be58d4f93cee admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.175325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.176196 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.176476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.176757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] PATH_INFO: `/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.177964 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c0e6f986ed4e91a5854fa4e26da6cd)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.180205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:26.322872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:26.324350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c0e6f986ed4e91a5854fa4e26da6cd', 'request_id': 'req-6cb64ae1-4439-49c4-9773-5340332a165d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:26.331159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c0e6f986ed4e91a5854fa4e26da6cd', 'request_id': 'req-6cb64ae1-4439-49c4-9773-5340332a165d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:26.391492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c0e6f986ed4e91a5854fa4e26da6cd', 'request_id': 'req-6cb64ae1-4439-49c4-9773-5340332a165d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:26.403878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] The token cache is being invalidate because project 64c0e6f986ed4e91a5854fa4e26da6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:26.404258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:26.404837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6cb64ae1-4439-49c4-9773-5340332a165d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:26.406411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1212/2428] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:26 2022] DELETE /identity/v3/projects/64c0e6f986ed4e91a5854fa4e26da6cd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:26.411457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.411910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.412018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.431299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:26.432672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:26.438596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:26.493244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1217/2429] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:26.501771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.502018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.502427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.521852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:26.523058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:26.532167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1213/2430] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:26.536156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.548906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aefc5-81c5-493f-9951-e92bf93e5e7d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.554139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.563342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:26.573248 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1218/2431] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:26.577882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.587712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f011c7f-a99a-4dc7-8fd3-a2f217239874 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.592925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.593666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.593950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.594227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.594928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.596954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:26.716669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1214/2432] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:26 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:26.718163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.728188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745b9ee-c9d3-422d-bb25-9d3439fd5135 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.787041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.787705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.787989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.788268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.788959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.790953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:26.908813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1219/2433] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:26 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:26.912820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.933476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd638-0d2d-4f47-bee6-77b1305e9c8b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.938988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.939661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.939954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.940231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.940960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.942593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:26.959164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1215/2434] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:56:26.960909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:26.976251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda68fd-ed56-486c-9491-1eca70acbb06 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:26.986813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:26.988022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:26.988512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:26.988829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:26.989612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:26.992231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e688c7-2bce-4789-94a4-025194f943bc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.005858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1220/2435] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:27.009920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.022390 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602502b-b857-4e31-97d7-8a6c8c9df69f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.028631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.029648 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.030283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.030283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] PATH_INFO: `/v3/projects/1d99d76b58284ae5bdac263e7eaf6967/users/22ab7f6aa1634ae29b6a2a540b52886e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.052959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d99d76b58284ae5bdac263e7eaf6967, user_id=22ab7f6aa1634ae29b6a2a540b52886e, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.055971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.127064 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1216/2436] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:27 2022] PUT /identity/v3/projects/1d99d76b58284ae5bdac263e7eaf6967/users/22ab7f6aa1634ae29b6a2a540b52886e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:27.139632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.139925 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.140206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.167324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] MFA Rules not processed for user `22ab7f6aa1634ae29b6a2a540b52886e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:27.168489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:27.234471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1221/2437] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:27.238930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.301915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75836-0bce-478a-b95e-9ac71a553b43 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.306889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.307537 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.307817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.308090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.308782 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.311930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.428121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1217/2438] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:27 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:27.432250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.443629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c58bf-8127-487d-814f-acf4de7e0bd0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.447198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:27.500650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.501346 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.501629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.501945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.502635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.504543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.596958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1222/2439] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:27 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:27.600973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.612014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066813-d56a-43ab-9e97-91c2fdf4db7d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.616669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.617350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.617639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.617916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.618871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.620463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.631838 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1218/2440] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:27.638272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.648417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aad62-3774-407a-85df-a1183c2e57ad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.653051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.654152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.654398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.654617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] PATH_INFO: `/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609/users/2dddd1f083cf489ea61602ecb4bc34b6/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.669336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcc1abfd4744568b9b7cf3f7ed0a609, user_id=2dddd1f083cf489ea61602ecb4bc34b6, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.672593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.745935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1223/2441] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:27 2022] PUT /identity/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609/users/2dddd1f083cf489ea61602ecb4bc34b6/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:27.750104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.759992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eac6-4f58-4aa4-a52a-866de1c47ce6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.805228 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:27.817948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.818640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.818945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.819202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.819918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.821556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88565cc-0e70-4f4b-9a21-8eb2606addb4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.831230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1219/2442] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:27.835397 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:27.846493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0b48-594c-4476-a169-3f0267c7eb73 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:27.851028 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:27.851891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.852171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.852441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] PATH_INFO: `/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609/users/2dddd1f083cf489ea61602ecb4bc34b6/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.854390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcc1abfd4744568b9b7cf3f7ed0a609, user_id=2dddd1f083cf489ea61602ecb4bc34b6, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:27.859747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:27.868077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1224/2443] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:27 2022] PUT /identity/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609/users/2dddd1f083cf489ea61602ecb4bc34b6/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:27.932890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:27.933214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:27.933489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:27.960783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] MFA Rules not processed for user `2dddd1f083cf489ea61602ecb4bc34b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:27.962163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:28.035182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1220/2444] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:28.046577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:28.048369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:28.109391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43746681-6f55-48a8-887e-b12d7296cc50 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:28.113172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:28.113888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:28.114237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:28.114578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:28.123969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:28.125548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:28.135435 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1225/2445] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:28.184628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:28.188891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:28.202316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ef5d-c4d9-46b9-b434-7e7001259617 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:28.206627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:28.207295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:28.207574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:28.207842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:28.268708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:28.270275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65044cb5-f7e2-4404-baa6-6eea6a92bb32 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:28.279686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1221/2446] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:33.086001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:33.160807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4538a-2394-49ff-b448-a5a97b9d95f1 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:33.166512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:33.167754 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:33.168046 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:33.173989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] PATH_INFO: `/v3/users/22ab7f6aa1634ae29b6a2a540b52886e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:33.177290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ab7f6aa1634ae29b6a2a540b52886e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:33.179385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:33.184354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:33.366086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ab7f6aa1634ae29b6a2a540b52886e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:33.373064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ab7f6aa1634ae29b6a2a540b52886e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:33.385841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ab7f6aa1634ae29b6a2a540b52886e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:33.386550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ab7f6aa1634ae29b6a2a540b52886e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:33.500328 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ab7f6aa1634ae29b6a2a540b52886e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:33.507125 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1226/2447] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:33 2022] DELETE /identity/v3/users/22ab7f6aa1634ae29b6a2a540b52886e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:33.511795 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:33.512280 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:33.512402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:33.531716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:33.532921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:33.597912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1222/2448] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:33.609367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:33.613057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:33.689699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded44c-39f9-4a09-b339-cc22497d9bed admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:33.693469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:33.694123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:33.694410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:33.694689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:33.704343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:33.705833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:33.715860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1227/2449] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:56:34.157799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:34.159227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:34.182288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab486c9-c311-4b68-b7d0-dec1fa3dcf9f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:34.188111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:34.188998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:34.189330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:34.189624 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] PATH_INFO: `/v3/projects/1d99d76b58284ae5bdac263e7eaf6967` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:34.190796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d99d76b58284ae5bdac263e7eaf6967)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:34.192802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:34.194489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:34.325576 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d99d76b58284ae5bdac263e7eaf6967', 'request_id': 'req-f64d4a12-9786-474e-ac0a-239b6adf68e2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.332503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d99d76b58284ae5bdac263e7eaf6967', 'request_id': 'req-f64d4a12-9786-474e-ac0a-239b6adf68e2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.394419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d99d76b58284ae5bdac263e7eaf6967', 'request_id': 'req-f64d4a12-9786-474e-ac0a-239b6adf68e2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.405961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] The token cache is being invalidate because project 1d99d76b58284ae5bdac263e7eaf6967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:34.406190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.406823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.408355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1223/2450] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:34 2022] DELETE /identity/v3/projects/1d99d76b58284ae5bdac263e7eaf6967 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:34.412509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:34.422076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca365-6b41-47e2-8d82-787c841acbde service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:34.477788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:34.478676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:34.478946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:34.479226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] PATH_INFO: `/v3/users/2dddd1f083cf489ea61602ecb4bc34b6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:34.480411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dddd1f083cf489ea61602ecb4bc34b6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:34.482353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:34.579567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:34.711227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dddd1f083cf489ea61602ecb4bc34b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.718038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dddd1f083cf489ea61602ecb4bc34b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.730589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dddd1f083cf489ea61602ecb4bc34b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.731270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dddd1f083cf489ea61602ecb4bc34b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.817232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b5f2c531-bfd8-493e-b635-095ae5630ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dddd1f083cf489ea61602ecb4bc34b6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:34.823471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1228/2451] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:34 2022] DELETE /identity/v3/users/2dddd1f083cf489ea61602ecb4bc34b6 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:35.213406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:35.214855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:35.235427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d4a12-9786-474e-ac0a-239b6adf68e2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:35.296024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:35.296885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.297217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.297503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] PATH_INFO: `/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.298690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcc1abfd4744568b9b7cf3f7ed0a609)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:35.300596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:35.454123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcc1abfd4744568b9b7cf3f7ed0a609', 'request_id': 'req-e436938c-a9df-44e9-a26c-c83ae222f844'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:35.460458 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcc1abfd4744568b9b7cf3f7ed0a609', 'request_id': 'req-e436938c-a9df-44e9-a26c-c83ae222f844'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:35.516825 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcc1abfd4744568b9b7cf3f7ed0a609', 'request_id': 'req-e436938c-a9df-44e9-a26c-c83ae222f844'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:35.527359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] The token cache is being invalidate because project 3fcc1abfd4744568b9b7cf3f7ed0a609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:35.528199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:35.528347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e436938c-a9df-44e9-a26c-c83ae222f844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:35.530392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1224/2452] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:35 2022] DELETE /identity/v3/projects/3fcc1abfd4744568b9b7cf3f7ed0a609 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:35.535241 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.535738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.535846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.556116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:35.557411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:35.616956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1229/2453] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:35.624850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.625177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.625457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.644022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:35.645310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:35.653001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1225/2454] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:35.658062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:35.668158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c019c-1300-4daf-be56-fb4086f4cfc9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:35.673013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:35.673726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.674008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.674282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.675002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:35.675307 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:35.676904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:35.690050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1230/2455] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:56:35.694107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:35.704581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd637f-1aab-45f2-a5b0-7553b91fe1ef None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:35.709234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:35.709872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.710151 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.710419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.711108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:35.712992 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:35.726593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:35.824697 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1226/2456] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:35 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:35.829599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:35.839609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2df1-93c5-4dcf-afb2-21b5f0339035 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:35.895126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676306-39d8-4723-8422-633812307ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:35.895788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676306-39d8-4723-8422-633812307ede admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:35.896074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676306-39d8-4723-8422-633812307ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:35.896348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f676306-39d8-4723-8422-633812307ede admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:35.897027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f676306-39d8-4723-8422-633812307ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:35.898943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f676306-39d8-4723-8422-633812307ede admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.000883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1231/2457] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:35 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:36.004842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.016120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dd56-ef15-441f-b0cd-747c8ba1b0c6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.020677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.021356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.021637 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.021911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.022629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.024209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.034428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1227/2458] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:36.038423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676306-39d8-4723-8422-633812307ede admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.048286 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f676306-39d8-4723-8422-633812307ede admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.052821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.053511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.053788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.054060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.055025 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.056589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6b160-0296-4e8b-b807-d37cddef97b1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.066886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1232/2459] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:36.070933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.081824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb0b23-84e2-4068-8072-5da3852f325c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.086319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.087181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.087461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.087734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] PATH_INFO: `/v3/projects/45d16ca2c94a4c2db2833cd220bad9cf/users/eef7a64c086a4b4eb3912e41986a1731/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.102447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d16ca2c94a4c2db2833cd220bad9cf, user_id=eef7a64c086a4b4eb3912e41986a1731, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.105065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.174760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1228/2460] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:36 2022] PUT /identity/v3/projects/45d16ca2c94a4c2db2833cd220bad9cf/users/eef7a64c086a4b4eb3912e41986a1731/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:56:36.187536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.187825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.188099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.207380 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:36.215884 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] MFA Rules not processed for user `eef7a64c086a4b4eb3912e41986a1731`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:36.217554 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:36.280883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1233/2461] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:36.285523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.352802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bda440-573c-4c3c-8267-353d4ee41498 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.358014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.358702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.358979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.359218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.359918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.361839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.446447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1229/2462] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:36 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:36.447816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.460027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8620f3-f65d-473e-89b9-acc555593533 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.516525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.517219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.517506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.517783 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.518467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.520371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.634904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1234/2463] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:36 2022] POST /identity/v3/users => generated 306 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:36.638909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.649610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58048a15-6799-4fc7-b284-2fe516a548b6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.654182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.654933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.655105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.655378 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.656153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.657735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.668605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1230/2464] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:36.673296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.683482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415669d4-5b51-45e5-9b0c-ea4f8b385fd7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.688939 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.689860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.690139 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.690411 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] PATH_INFO: `/v3/projects/6917cc3078e04e7780e91124b87f3b6d/users/08f0fed35c994fcd9a0d11eb1c5ce16e/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.705339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6917cc3078e04e7780e91124b87f3b6d, user_id=08f0fed35c994fcd9a0d11eb1c5ce16e, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.707972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.789378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1235/2465] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:36 2022] PUT /identity/v3/projects/6917cc3078e04e7780e91124b87f3b6d/users/08f0fed35c994fcd9a0d11eb1c5ce16e/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:36.793236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.804258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4e542-9205-42ea-b736-fe243bf87c07 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.861511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.862172 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.862453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.862725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.863438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.865008 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1aff7-0e77-473a-8f2f-d04223b1ef2f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.874656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1231/2466] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:36.878795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:36.889201 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983024d-6d47-42a8-9a89-d56a8c73d999 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:36.893797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:36.894666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.894946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.895220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] PATH_INFO: `/v3/projects/6917cc3078e04e7780e91124b87f3b6d/users/08f0fed35c994fcd9a0d11eb1c5ce16e/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.897210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6917cc3078e04e7780e91124b87f3b6d, user_id=08f0fed35c994fcd9a0d11eb1c5ce16e, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:36.899824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:36.910477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1236/2467] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:36 2022] PUT /identity/v3/projects/6917cc3078e04e7780e91124b87f3b6d/users/08f0fed35c994fcd9a0d11eb1c5ce16e/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:36.921592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:36.921878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:36.922152 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:36.948801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] MFA Rules not processed for user `08f0fed35c994fcd9a0d11eb1c5ce16e`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:36.950037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:36.962213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:37.015980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1232/2468] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:37.026811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:37.028541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:37.087325 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00fdb-1229-41e9-873d-30986e77ce41 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:37.090929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:37.091562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:37.093021 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:37.093337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:37.099150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:37.102602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:37.104015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:37.114178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1237/2469] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:37.358027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:37.367963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41482-babc-4ed2-84d2-5778b60552df None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:37.371668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:37.372312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:37.372597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:37.372870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:37.433824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:37.435194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:37.444978 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1233/2470] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:41.189696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:41.191117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:41.256837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df446af0-89a7-460f-8add-2e452d90f52f service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:41.260693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:41.261364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:41.261710 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:41.261997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:41.263789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:41.273763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:41.275230 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9765-ef3a-4fdf-abd7-46c66329a39f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:41.283898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1238/2471] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:42.622150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:42.690321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19050f-d89c-4392-b298-88aef82feae7 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:42.695685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:42.705102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:42.705102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:42.705102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] PATH_INFO: `/v3/users/eef7a64c086a4b4eb3912e41986a1731` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:42.705102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef7a64c086a4b4eb3912e41986a1731)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:42.705102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:42.915364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef7a64c086a4b4eb3912e41986a1731'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:42.921068 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:42.929334 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef7a64c086a4b4eb3912e41986a1731'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:42.932846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef7a64c086a4b4eb3912e41986a1731'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:42.932846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef7a64c086a4b4eb3912e41986a1731'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:42.937892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef7a64c086a4b4eb3912e41986a1731'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.042445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1234/2472] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:42 2022] DELETE /identity/v3/users/eef7a64c086a4b4eb3912e41986a1731 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:43.048188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:43.048687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:43.048802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:43.066548 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:43.070390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:43.127338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1239/2473] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:43.136933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:43.138359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:43.197329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ac234-b893-4537-bfd2-636ea08eceb2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:43.201184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:43.201870 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:43.202647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:43.203001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:43.212158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:43.213640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:43.224141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1235/2474] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:43.687024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:43.688327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:43.701721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71716a-ed27-4de4-a32b-4f7b50fa4748 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:43.708655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:43.709920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:43.715208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:43.715208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] PATH_INFO: `/v3/projects/45d16ca2c94a4c2db2833cd220bad9cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:43.715208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d16ca2c94a4c2db2833cd220bad9cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:43.715208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:43.827757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d16ca2c94a4c2db2833cd220bad9cf', 'request_id': 'req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.852193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d16ca2c94a4c2db2833cd220bad9cf', 'request_id': 'req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.943686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d16ca2c94a4c2db2833cd220bad9cf', 'request_id': 'req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.962940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] The token cache is being invalidate because project 45d16ca2c94a4c2db2833cd220bad9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:43.963229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.964469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:43.966798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1240/2475] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:43 2022] DELETE /identity/v3/projects/45d16ca2c94a4c2db2833cd220bad9cf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:43.970753 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:43.980396 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d820-bf32-4a87-b4bc-874cd7197068 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:44.037124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:44.037985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:44.038269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:44.038542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] PATH_INFO: `/v3/users/08f0fed35c994fcd9a0d11eb1c5ce16e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:44.040673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f0fed35c994fcd9a0d11eb1c5ce16e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:44.042614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:44.231755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f0fed35c994fcd9a0d11eb1c5ce16e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:44.244167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f0fed35c994fcd9a0d11eb1c5ce16e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:44.244951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f0fed35c994fcd9a0d11eb1c5ce16e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:44.245585 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f0fed35c994fcd9a0d11eb1c5ce16e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:44.252582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-baa5c6cf-b07c-42f5-8a8e-1ed259e65e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f0fed35c994fcd9a0d11eb1c5ce16e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:44.254284 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:44.319828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1236/2476] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:56:43 2022] DELETE /identity/v3/users/08f0fed35c994fcd9a0d11eb1c5ce16e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:45.199580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.201169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:45.214068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c1be6-b16b-4bfc-8c7d-09ec952a1473 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.236504 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:45.294270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.295135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.295413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.295682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] PATH_INFO: `/v3/projects/6917cc3078e04e7780e91124b87f3b6d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.297640 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6917cc3078e04e7780e91124b87f3b6d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:45.299556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:45.444950 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6917cc3078e04e7780e91124b87f3b6d', 'request_id': 'req-92798573-4044-48d6-a6bc-cf641afe4ec6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:45.451293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6917cc3078e04e7780e91124b87f3b6d', 'request_id': 'req-92798573-4044-48d6-a6bc-cf641afe4ec6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:45.511294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6917cc3078e04e7780e91124b87f3b6d', 'request_id': 'req-92798573-4044-48d6-a6bc-cf641afe4ec6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:45.523014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] The token cache is being invalidate because project 6917cc3078e04e7780e91124b87f3b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:56:45.523299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:45.524555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92798573-4044-48d6-a6bc-cf641afe4ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:56:45.526123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1241/2477] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:56:45 2022] DELETE /identity/v3/projects/6917cc3078e04e7780e91124b87f3b6d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:45.532013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.532494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.532599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.550747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:45.552062 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:45.615000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1237/2478] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:45.622871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.623165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.623495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.642537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:45.643866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:45.652198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1242/2479] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:56:45.656729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.666494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b14130-d542-4e5d-b837-1c1633d3aaea None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.671786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.672542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.672848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.673162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.673899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:56:45.674210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:45.675796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:45.688545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1238/2480] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:56:45.692805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.702723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb455161-cf9e-453a-9645-de2bfd3511a6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.707436 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.708088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.708372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.708645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.709373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:45.711272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:45.797391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1243/2481] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:56:45.801679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.811997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a084df-53d2-4f2e-9dae-82f13968c130 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.867930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.868594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.868878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.869187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.869872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:45.871781 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:45.943279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1239/2482] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:56:45 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:56:45.947329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.958332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a7ea-5c8a-46ba-9bab-3b00bb3c387d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.962922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.963618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.963895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.964166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.964882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:45.966491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:45.978100 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1244/2483] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:45.981926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:45.992663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dd545-07f7-4617-8b2c-420d42a007a1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:45.997263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:45.997908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:45.998187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:45.998456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:45.999499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:46.001116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71bfab6-3069-4a09-b20c-00e5ff8f6625 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:46.010792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1240/2484] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:56:46.014788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:46.024735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02a59-e63a-4763-9d38-f69980c1d6a5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:46.029349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:46.030219 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:46.030996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:46.031272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] PATH_INFO: `/v3/projects/10e892abdc844e02b9253764bdc90be3/users/d1553325e53a4c46b287defa3a04726e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:46.045555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e892abdc844e02b9253764bdc90be3, user_id=d1553325e53a4c46b287defa3a04726e, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:46.048880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:46.146659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1245/2485] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:56:46 2022] PUT /identity/v3/projects/10e892abdc844e02b9253764bdc90be3/users/d1553325e53a4c46b287defa3a04726e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:56:46.158703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:46.158999 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:46.159315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:46.187880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] MFA Rules not processed for user `d1553325e53a4c46b287defa3a04726e`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:56:46.189115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:46.233311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:46.252295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1241/2486] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:56:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:56:46.263475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:56:46.265814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:56:46.322562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:46.328166 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689192-6449-4f9c-85de-d8748defa651 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:56:46.332058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:56:46.332706 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:56:46.332988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:56:46.336845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:56:46.341771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:56:46.344093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:56:46.357649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1246/2487] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:56:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:56:56.329375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:56:56.486040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:02.028722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:57:02.030176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:02.123810 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34788d9e-71fa-4774-9627-7a0379fdc33b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:57:02.129154 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:57:02.129813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:02.130110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:02.130383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:02.132142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:02.143532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:57:02.145192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c2a3b-8de4-48bd-b31e-7eba286444be service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:57:02.155647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1242/2488] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:57:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:57:12.150932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=41 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:15.452526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:57:15.516039 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed9558-3c57-43a8-9327-dc3866568f23 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:57:15.520687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:57:15.521587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:15.521866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:15.522138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] PATH_INFO: `/v3/users/d1553325e53a4c46b287defa3a04726e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:15.523315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1553325e53a4c46b287defa3a04726e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:57:15.525318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:57:15.711594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1553325e53a4c46b287defa3a04726e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:15.719104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1553325e53a4c46b287defa3a04726e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:15.734895 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1553325e53a4c46b287defa3a04726e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:15.735803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1553325e53a4c46b287defa3a04726e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:15.826390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1553325e53a4c46b287defa3a04726e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:15.835014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1247/2489] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:57:15 2022] DELETE /identity/v3/users/d1553325e53a4c46b287defa3a04726e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:57:15.840411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:15.840787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:15.840887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:15.874112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:57:15.875956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:15.950841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1243/2490] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:57:15.961540 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:57:15.962954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:16.028798 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ca9-0f54-4792-b8bd-caf330250fe2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:57:16.032890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:57:16.034849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:16.035142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:16.035417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:16.044889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:57:16.046541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d875a-15e3-4c5f-b9f0-39e9d50c010c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:57:16.058221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1248/2491] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:57:16.523183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:57:16.524759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:16.539093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cefbe-dc38-49b6-96c4-849976bec595 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:57:16.545162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:57:16.551986 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:16.552323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:16.552666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] PATH_INFO: `/v3/projects/10e892abdc844e02b9253764bdc90be3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:16.554475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e892abdc844e02b9253764bdc90be3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:57:16.557189 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:57:16.695963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e892abdc844e02b9253764bdc90be3', 'request_id': 'req-958b778c-fdfd-4cac-8dc3-63f685fb7c30'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:16.703041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e892abdc844e02b9253764bdc90be3', 'request_id': 'req-958b778c-fdfd-4cac-8dc3-63f685fb7c30'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:16.769879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e892abdc844e02b9253764bdc90be3', 'request_id': 'req-958b778c-fdfd-4cac-8dc3-63f685fb7c30'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:16.781594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] The token cache is being invalidate because project 10e892abdc844e02b9253764bdc90be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:57:16.781878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:16.782490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-958b778c-fdfd-4cac-8dc3-63f685fb7c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:57:16.785275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1244/2492] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:57:16 2022] DELETE /identity/v3/projects/10e892abdc844e02b9253764bdc90be3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:57:20.445114 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:20.445628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:20.445742 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:20.463053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] MFA Rules not processed for user `628e662a3b49438f9930a2d160658b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:57:20.464226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:20.481096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:20.517954 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1249/2493] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:57:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:57:26.778244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:30.592367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=15 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:47.880358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304120-b7cd-42f8-8002-d4101a3f0835 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:47.891937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304120-b7cd-42f8-8002-d4101a3f0835 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:47.891937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304120-b7cd-42f8-8002-d4101a3f0835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:47.900804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c1304120-b7cd-42f8-8002-d4101a3f0835 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:57:47.902058 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c1304120-b7cd-42f8-8002-d4101a3f0835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:47.964516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1245/2494] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:57:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:57:48.102453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:57:48.103890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:57:48.186431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f87e-df02-47bb-95cf-08820f7a82c0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:57:48.191426 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:57:48.192073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:57:48.192354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:57:48.193245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:57:48.201494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:57:48.203099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:57:48.211916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1250/2495] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:57:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:57:57.957136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=28 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:57:58.207810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=32 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:58:08.200671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fcc8-396d-49f3-89b5-f1cbc6db50c7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:58:08.201155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fcc8-396d-49f3-89b5-f1cbc6db50c7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:58:08.201261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fcc8-396d-49f3-89b5-f1cbc6db50c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:58:08.219025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4596fcc8-396d-49f3-89b5-f1cbc6db50c7 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:58:08.220229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4596fcc8-396d-49f3-89b5-f1cbc6db50c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:58:08.227643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1246/2496] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 12:58:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:58:08.239600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:58:08.250602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196d44d-5966-4eea-96bc-fadeed44176b service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:58:08.255621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:58:08.256247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:58:08.256547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:58:08.256805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:58:08.265913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:58:08.267515 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:58:08.275663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1251/2497] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:58:18.213722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:58:18.272295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:58:29.543178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:58:29.547767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:58:29.547767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:58:29.562061 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:58:29.563265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:58:29.570750 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1247/2498] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 12:58:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:58:29.584291 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:58:29.595379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b363844-6887-4213-a86c-927ed99da3a8 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:58:29.600703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:58:29.601382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:58:29.601667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:58:29.601945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:58:29.610608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:58:29.612164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:58:29.620447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1252/2499] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 12:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 12:58:39.556760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:58:39.617155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:58:56.711530 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:58:56.712912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:58:56.768463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f1eb0-346b-4eff-902a-c25894dfbf95 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:58:56.772027 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:58:56.772669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:58:56.772982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:58:56.773292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:58:56.775019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:58:56.837361 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:58:56.838812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:58:56.849181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1248/2500] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:00.337823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:00.339795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:00.411998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ab67-dcb0-4b5b-9a49-475b46ee504d service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:00.417564 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:00.418274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:00.418559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:00.418833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:00.421991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:00.437872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:00.439302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:00.439854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:00.440981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:00.452294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1253/2501] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:00.458660 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cfc84-2cbc-4f72-bdf5-fb1ae044ba8e service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:00.463497 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:00.464500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:00.464852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:00.465300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] PATH_INFO: `/v3/users/46b5c80d8bf34965a30b9b67e9117996` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:00.492951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b5c80d8bf34965a30b9b67e9117996)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:00.494991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:00.648605 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b5c80d8bf34965a30b9b67e9117996'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:00.661175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b5c80d8bf34965a30b9b67e9117996'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:00.661839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b5c80d8bf34965a30b9b67e9117996'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:00.662511 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b5c80d8bf34965a30b9b67e9117996'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:00.669384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c70d84-5bbd-46a9-8ada-88d2ee235d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b5c80d8bf34965a30b9b67e9117996'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:00.743165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1249/2502] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:00 2022] DELETE /identity/v3/users/46b5c80d8bf34965a30b9b67e9117996 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:01.001784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.014674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e82131-7889-4e97-89a0-156b33694fd1 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.075115 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.075966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.076496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.076496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] PATH_INFO: `/v3/projects/858d6855277f465497c27313765e5d1a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.087082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] RBAC: Authorizing `identity:delete_project(project_id=858d6855277f465497c27313765e5d1a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.089056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.195213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858d6855277f465497c27313765e5d1a', 'request_id': 'req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:01.201885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858d6855277f465497c27313765e5d1a', 'request_id': 'req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:01.269024 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858d6855277f465497c27313765e5d1a', 'request_id': 'req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:01.280674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] The token cache is being invalidate because project 858d6855277f465497c27313765e5d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:01.283994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:01.283994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5293181c-66d1-4fb2-b5b0-e1f9bf77563b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:01.283994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1254/2503] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:01 2022] DELETE /identity/v3/projects/858d6855277f465497c27313765e5d1a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:01.288125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.288425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.288707 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.307724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:01.308920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:01.314281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:01.367850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1250/2504] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:01.376006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.380003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.380003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.394261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:01.395471 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:01.403351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1255/2505] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:01.408098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.417631 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d964a-e4f5-473c-9304-ec507e6717cd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.423086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.423733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.424013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.424279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.424991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:01.425323 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.427003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.439744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1251/2506] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:01.444001 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.449615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:01.454081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a1b-dcc6-485a-b4e7-f939822ae86c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.458676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.466067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.466067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.466067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.466067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.466067 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.538876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1256/2507] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:01 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:01.543141 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.553466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8da34-3b4a-48aa-9a63-3e6a429ca6c8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.609665 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.610339 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.610625 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.610892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.612194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.614144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.691617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1252/2508] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:01 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:01.695651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.706417 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68e144-f261-41ce-89b5-efa41b5db241 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.711077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.713637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.713637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.713637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.713637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.714659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.726035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1257/2509] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:01.730070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.740331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe452492-f214-476c-bf41-ef750032f36d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.745055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.745809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.746100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.746377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.747100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.748677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f47a20-874d-41e7-b24e-7e4cada630c0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.758823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1253/2510] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:01.762975 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.773334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef9a9b-df9c-4341-9bf1-2aaa7d7116c6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:01.777935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:01.778806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.779089 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.780016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] PATH_INFO: `/v3/projects/185422595a1e4037afa71e49c8837f96/users/ec4bec9d38704e4b92ad06336f3a7a3c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.794507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=185422595a1e4037afa71e49c8837f96, user_id=ec4bec9d38704e4b92ad06336f3a7a3c, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:01.797264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:01.862010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1258/2511] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:01 2022] PUT /identity/v3/projects/185422595a1e4037afa71e49c8837f96/users/ec4bec9d38704e4b92ad06336f3a7a3c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:01.872308 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:01.872594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:01.873645 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:01.901270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] MFA Rules not processed for user `ec4bec9d38704e4b92ad06336f3a7a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:01.902466 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:01.966941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1254/2512] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:01.977624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:01.979316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:02.072717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949dd6f-d4c3-4a9b-bfc4-e0ad4c134bee admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:02.078874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:02.079509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:02.079787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:02.080057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:02.087791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:02.089384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:02.098034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1259/2513] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:02.117793 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:02.119182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:02.129538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:02.177135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2a36-cc68-47d3-bfbc-579fcbe41403 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:02.181186 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:02.182266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:02.182266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:02.182463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:02.184217 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:02.199156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:02.200575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:02.212229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1255/2514] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:02.901753 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:02.911937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dd069-503e-4610-a21d-3841d8bfb300 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:02.915497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:02.916127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:02.916429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:02.916663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:02.918760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:02.933468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:02.935335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa16385-d138-4bd3-8927-466a65652c36 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:02.946369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1260/2515] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:12.207454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=23 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:12.501527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:12.560629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fafc2-78bc-4b37-94e9-f58b5e732c0c service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:12.565732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:12.566898 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:12.567176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:12.567448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] PATH_INFO: `/v3/users/ec4bec9d38704e4b92ad06336f3a7a3c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:12.568787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4bec9d38704e4b92ad06336f3a7a3c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:12.570751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:12.726514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4bec9d38704e4b92ad06336f3a7a3c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:12.740102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4bec9d38704e4b92ad06336f3a7a3c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:12.740864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4bec9d38704e4b92ad06336f3a7a3c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:12.741485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4bec9d38704e4b92ad06336f3a7a3c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:12.748440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4bec9d38704e4b92ad06336f3a7a3c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:12.834905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1256/2516] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:12 2022] DELETE /identity/v3/users/ec4bec9d38704e4b92ad06336f3a7a3c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:12.839486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:12.839962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:12.840057 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:12.857765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:12.858953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:12.909049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:12.917311 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1261/2517] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:12.928016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:12.929457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:12.987740 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849b7df-8d2a-4b3d-9d5c-786a7574746e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:12.991469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:12.992110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:12.992391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:12.992664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.001619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:13.003221 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51336a54-b2dc-4613-976d-b6c203b90c2e service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:13.012734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1257/2518] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:13.465093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:13.466596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:13.479584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058d76-f162-4cc9-9970-5c858a0ac3c5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:13.484232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:13.485123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:13.485412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:13.485688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] PATH_INFO: `/v3/projects/185422595a1e4037afa71e49c8837f96` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.486885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=185422595a1e4037afa71e49c8837f96)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:13.489282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:13.592984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185422595a1e4037afa71e49c8837f96', 'request_id': 'req-39b80df0-378a-4ef2-b5aa-374535a65e4a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:13.599789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185422595a1e4037afa71e49c8837f96', 'request_id': 'req-39b80df0-378a-4ef2-b5aa-374535a65e4a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:13.689636 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185422595a1e4037afa71e49c8837f96', 'request_id': 'req-39b80df0-378a-4ef2-b5aa-374535a65e4a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:13.700404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] The token cache is being invalidate because project 185422595a1e4037afa71e49c8837f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:13.700691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:13.701403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-39b80df0-378a-4ef2-b5aa-374535a65e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:13.703004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1262/2519] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:13 2022] DELETE /identity/v3/projects/185422595a1e4037afa71e49c8837f96 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:13.708852 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:13.709353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:13.709454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.728397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:13.729627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:13.746611 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:13.792435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1258/2520] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:13.801440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:13.801837 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:13.802225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.821583 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:13.822790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:13.831476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1263/2521] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:13.835714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:13.845335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eaf63-5c06-4325-81b6-278b3b834bc3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:13.850047 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:13.850721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:13.851000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:13.851270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.851989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:13.852288 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:13.853896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:13.866200 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1259/2522] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:59:13.870409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:13.880432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc1c55-7d86-4cd9-a520-ce97dc58eaae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:13.885157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:13.885812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:13.886098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:13.886431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:13.887141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:13.889070 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:13.993725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1264/2523] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:13 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:13.997680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.008418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d347bc-d80c-45bc-b6a9-ec5279ae1862 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.064492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.065659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.066120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.066517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.067203 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.069123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:14.171563 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1260/2524] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:13 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:14.177634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.187879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1a832-2ce1-4c3e-bcc0-f667d5eb193f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.193355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.194031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.194305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.194578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.195295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.196886 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:14.207408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1265/2525] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:14.211380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.221422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dac8a-ec41-40b7-9e3c-3d25c25e0f6e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.226290 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.226948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.227234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.227513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.228235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.229884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11308ba7-cfba-46e1-9fa6-24ac039f7a25 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:14.240522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1261/2526] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:14.244340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.254822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c2698-6aae-49d9-a758-ddb7f7306b3a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.259496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.260392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.260649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.260946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] PATH_INFO: `/v3/projects/7062b770b1144534931c2357c27e15ba/users/d44cb290ef8c4b1a96d4e918d6c3468b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.277006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7062b770b1144534931c2357c27e15ba, user_id=d44cb290ef8c4b1a96d4e918d6c3468b, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.279689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:14.349933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1266/2527] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:14 2022] PUT /identity/v3/projects/7062b770b1144534931c2357c27e15ba/users/d44cb290ef8c4b1a96d4e918d6c3468b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:14.360127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.360424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.360704 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.390414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] MFA Rules not processed for user `d44cb290ef8c4b1a96d4e918d6c3468b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:14.391630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:14.457984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1262/2528] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:14.469627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.471386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:14.551919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c649-0696-4a32-946c-0ef80e3d47e6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.557714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.558632 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.559084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.559559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.568266 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.569921 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bac73-fa00-4bcf-a617-05eee2364a80 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:14.579377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1267/2529] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:14.813941 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:14.837107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:14.890023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d18bf-7a66-40bc-bbf4-bb33531a12dd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:14.897715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:14.899167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:14.899469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:14.900398 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] PATH_INFO: `/v3/users/d44cb290ef8c4b1a96d4e918d6c3468b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:14.900927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44cb290ef8c4b1a96d4e918d6c3468b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:14.902874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:15.106192 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44cb290ef8c4b1a96d4e918d6c3468b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:15.119603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44cb290ef8c4b1a96d4e918d6c3468b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:15.120375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44cb290ef8c4b1a96d4e918d6c3468b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:15.121190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44cb290ef8c4b1a96d4e918d6c3468b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:15.128854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44cb290ef8c4b1a96d4e918d6c3468b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:15.195716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1263/2530] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:14 2022] DELETE /identity/v3/users/d44cb290ef8c4b1a96d4e918d6c3468b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:15.200272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:15.200765 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:15.200951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:15.211220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:15.223688 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:15.225131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:15.289946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1268/2531] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:15.300387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:15.301919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:15.359943 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585fa15-47bb-4404-a138-4b299d067645 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:15.364285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:15.364928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:15.365248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:15.365524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:15.374574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:15.375971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea0dc7-3875-4a05-ae2d-5f1ef8a054b6 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:15.385454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1264/2532] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:15.848520 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:15.850060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:15.867784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccb912-2bf5-47aa-b807-0393a8fdf985 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:15.881601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:15.882477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:15.882789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:15.883064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] PATH_INFO: `/v3/projects/7062b770b1144534931c2357c27e15ba` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:15.884293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7062b770b1144534931c2357c27e15ba)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:15.886295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.175780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7062b770b1144534931c2357c27e15ba', 'request_id': 'req-069a75b0-84a2-499a-a5fd-e85c1037338c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:16.182680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7062b770b1144534931c2357c27e15ba', 'request_id': 'req-069a75b0-84a2-499a-a5fd-e85c1037338c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:16.266690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7062b770b1144534931c2357c27e15ba', 'request_id': 'req-069a75b0-84a2-499a-a5fd-e85c1037338c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:16.278747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] The token cache is being invalidate because project 7062b770b1144534931c2357c27e15ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:16.279037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:16.279672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069a75b0-84a2-499a-a5fd-e85c1037338c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:16.281300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1269/2533] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:15 2022] DELETE /identity/v3/projects/7062b770b1144534931c2357c27e15ba => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:16.287683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.288153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.288257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.306313 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:16.307516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:16.341476 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:16.366916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1265/2534] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:16.374815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.375107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.375389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.394562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:16.395755 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:16.403760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1270/2535] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:16.407926 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.417609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd28a-6a48-491c-a5d0-02f892ad80fc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.422164 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.422814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.423106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.423369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.424097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:16.424405 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.427075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.439834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1266/2536] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:16.444758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.454051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a14be2-76a5-4783-a7b8-530a3d6e4d80 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.458888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.459535 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.459819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.460095 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.460788 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.462719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.542970 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1271/2537] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:16 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:16.547131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.558006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8461b-c283-4016-9c72-36c12cbe0b95 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.615399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.616059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.616338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.616609 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.617319 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.619257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.718919 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1267/2538] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:16 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:16.723576 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.734158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140858-9277-4e23-a341-791d6d3393a6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.740031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.740031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.740419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.740694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.741483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.743077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.753537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1272/2539] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:16.757477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.767602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002a3d-79a2-4dc9-8ed6-2789d2b0f462 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.772266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.772939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.773230 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.773507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.774235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.775843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3236c4-fcb8-4944-9234-2190efb88a63 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.786212 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1268/2540] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:16.790267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:16.800974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345a7c-28f3-4ab0-a627-7888822d489f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:16.805870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:16.806736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.807149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.807432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] PATH_INFO: `/v3/projects/612fb0c85cb34fcf86bda5ce22f3d933/users/717fe8a9910b44319af263ab521d6a34/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.822091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=612fb0c85cb34fcf86bda5ce22f3d933, user_id=717fe8a9910b44319af263ab521d6a34, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:16.824794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:16.943731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1273/2541] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:16 2022] PUT /identity/v3/projects/612fb0c85cb34fcf86bda5ce22f3d933/users/717fe8a9910b44319af263ab521d6a34/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:16.955739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:16.956190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:16.956633 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:16.986218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] MFA Rules not processed for user `717fe8a9910b44319af263ab521d6a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:16.987581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:17.056171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1269/2542] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:17.060572 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.088402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:17.125276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabac3d1-2ea9-4c07-a834-cfee736e7f7f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.130722 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.131442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.131732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.132302 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.132704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.134703 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.231707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1274/2543] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:17 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:17.232872 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.243845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99ab9-329e-40ef-966d-d0cba4e1838b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.281256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:17.307447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.308104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.308395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.308671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.309404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.311469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.417958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1270/2544] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:17 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:17.422049 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.433915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f908-c3c5-4f16-a5ed-ef7389991cf2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.438718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.439385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.439668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.439947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.440669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.442356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.453885 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1275/2545] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:17.458947 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.470590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38214-7c1f-45fd-96c2-5199ae7be572 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.477803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.479824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.479824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.479824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] PATH_INFO: `/v3/projects/592d91a4302840bb8c71c9c3714324a2/users/ac557e85fbbf438baed213b8359dbc5c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.495550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592d91a4302840bb8c71c9c3714324a2, user_id=ac557e85fbbf438baed213b8359dbc5c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.498286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.593365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1271/2546] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:17 2022] PUT /identity/v3/projects/592d91a4302840bb8c71c9c3714324a2/users/ac557e85fbbf438baed213b8359dbc5c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:17.602150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.608876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28d9c9-5a15-4d29-bbeb-802ee6f27a9f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.665018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.665707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.665990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.666267 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.666983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.668568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9224b-6953-4e04-8ebf-be4cd814c02d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.678794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1276/2547] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:17.683156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.693826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55b9b0-25af-4c25-b179-62774a1dced6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.698407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.699819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.700105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.700380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] PATH_INFO: `/v3/projects/592d91a4302840bb8c71c9c3714324a2/users/ac557e85fbbf438baed213b8359dbc5c/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.702354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=592d91a4302840bb8c71c9c3714324a2, user_id=ac557e85fbbf438baed213b8359dbc5c, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.704979 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.715180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1272/2548] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:17 2022] PUT /identity/v3/projects/592d91a4302840bb8c71c9c3714324a2/users/ac557e85fbbf438baed213b8359dbc5c/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:17.725696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.725990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.726269 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.756518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] MFA Rules not processed for user `ac557e85fbbf438baed213b8359dbc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:17.756518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:17.822342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1277/2549] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:17.833973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:17.835702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:17.891558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d81e25-51a8-497c-96f4-51108b16a1af admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:17.895701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:17.896384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:17.896658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:17.896935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:17.905949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:17.907373 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f765c7-646b-49a2-97e4-dc895c9d0f5b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:17.917309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1273/2550] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:18.908903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:18.936088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:18.977993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4a6cb-f769-4efe-b02c-5bd95c31e28e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:18.982544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:18.983414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:18.983696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:18.983965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] PATH_INFO: `/v3/users/717fe8a9910b44319af263ab521d6a34` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:18.985175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=717fe8a9910b44319af263ab521d6a34)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:18.987102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:19.188293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717fe8a9910b44319af263ab521d6a34'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:19.195525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717fe8a9910b44319af263ab521d6a34'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:19.208257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717fe8a9910b44319af263ab521d6a34'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:19.208952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717fe8a9910b44319af263ab521d6a34'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:19.257857 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717fe8a9910b44319af263ab521d6a34'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:19.264721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1278/2551] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:18 2022] DELETE /identity/v3/users/717fe8a9910b44319af263ab521d6a34 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:19.269422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:19.269895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:19.270265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:19.288808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:19.290080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:19.352493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1274/2552] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:19.362034 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:19.363423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:19.419447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0b6cf-70f9-4b30-8cd7-e14a53a63c82 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:19.423020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:19.423659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:19.423937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:19.424209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:19.433534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:19.434877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:19.444053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1279/2553] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:19.851800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:19.853300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:19.865928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bb03b-471b-4293-ab41-066f29c660bf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:19.870465 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:19.871322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:19.871603 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:19.871874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] PATH_INFO: `/v3/projects/612fb0c85cb34fcf86bda5ce22f3d933` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:19.873195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612fb0c85cb34fcf86bda5ce22f3d933)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:19.875114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:20.016490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:20.031819 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612fb0c85cb34fcf86bda5ce22f3d933', 'request_id': 'req-1c355d0c-f38d-466d-ab72-653ff033d8c9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.038646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612fb0c85cb34fcf86bda5ce22f3d933', 'request_id': 'req-1c355d0c-f38d-466d-ab72-653ff033d8c9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.108588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612fb0c85cb34fcf86bda5ce22f3d933', 'request_id': 'req-1c355d0c-f38d-466d-ab72-653ff033d8c9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.119180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] The token cache is being invalidate because project 612fb0c85cb34fcf86bda5ce22f3d933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:20.123490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.123490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.123490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1275/2554] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:19 2022] DELETE /identity/v3/projects/612fb0c85cb34fcf86bda5ce22f3d933 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:20.125713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:20.135555 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6592391-e73b-46f1-86c1-2dd6f4095e67 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:20.176205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:20.192822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:20.193794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:20.194076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:20.194348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] PATH_INFO: `/v3/users/ac557e85fbbf438baed213b8359dbc5c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:20.195527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac557e85fbbf438baed213b8359dbc5c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:20.197476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:20.388611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac557e85fbbf438baed213b8359dbc5c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.396159 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac557e85fbbf438baed213b8359dbc5c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.408158 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac557e85fbbf438baed213b8359dbc5c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.408881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac557e85fbbf438baed213b8359dbc5c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.486553 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ef5f7ec-a72c-4a19-b763-7873cf879d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac557e85fbbf438baed213b8359dbc5c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:20.494012 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1280/2555] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:20 2022] DELETE /identity/v3/users/ac557e85fbbf438baed213b8359dbc5c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:20.936928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:20.939126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:20.953540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c355d0c-f38d-466d-ab72-653ff033d8c9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.016108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.017049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.017340 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.017618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] PATH_INFO: `/v3/projects/592d91a4302840bb8c71c9c3714324a2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.018801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=592d91a4302840bb8c71c9c3714324a2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.020791 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.151055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592d91a4302840bb8c71c9c3714324a2', 'request_id': 'req-29b82026-13cc-4797-898c-0cedfddd62bc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:21.157890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592d91a4302840bb8c71c9c3714324a2', 'request_id': 'req-29b82026-13cc-4797-898c-0cedfddd62bc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:21.217718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592d91a4302840bb8c71c9c3714324a2', 'request_id': 'req-29b82026-13cc-4797-898c-0cedfddd62bc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:21.229718 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] The token cache is being invalidate because project 592d91a4302840bb8c71c9c3714324a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:21.230002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:21.230641 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-29b82026-13cc-4797-898c-0cedfddd62bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:21.232512 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1276/2556] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:20 2022] DELETE /identity/v3/projects/592d91a4302840bb8c71c9c3714324a2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:21.238317 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.238959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.239133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.258381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:21.259633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:21.322270 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1281/2557] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:21.330456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.330752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.331035 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.350255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:21.351469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:21.359668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1277/2558] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:21.363929 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.373865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34510f1-525e-4c4b-8437-02c8f889cc99 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.378544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.379206 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.380061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.380340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.381088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:21.381402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.383053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.394865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1282/2559] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:21.403985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.413758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59cfe5-9de1-4d3e-9afd-a81742194290 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.418652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.419307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.419594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.419865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.420551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.422485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.540698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1278/2560] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:21 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:21.541922 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.553271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8b79-09a3-45b5-b74e-e0d965d7b606 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.603653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:21.627352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.628053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.628341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.628623 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.629377 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.632104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.755887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1283/2561] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:21 2022] POST /identity/v3/users => generated 305 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:21.760205 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.770840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcd9ff-752a-49b3-b7f5-18a49e2f65e7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.775362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.782799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.782799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.782799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.782799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.782799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.789876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1279/2562] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:21.794018 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.805745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b74f40-fb78-4688-a6b8-59c372c6f85e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.810547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.811207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.811489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.811768 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.812496 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.814275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111c3c5-906a-44d7-a46a-fc7e693674f9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.826198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1284/2563] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:21.830468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:21.837074 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:21.841353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c038c6-3a0e-4df2-9553-194927fecdb6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:21.846303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:21.852025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.852025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.852025 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] PATH_INFO: `/v3/projects/b83750567fd84b7785ddc8db9f545331/users/ad2802f4404046df8daaf360777f7227/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:21.864582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83750567fd84b7785ddc8db9f545331, user_id=ad2802f4404046df8daaf360777f7227, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:21.867503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:21.956139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1280/2564] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:21 2022] PUT /identity/v3/projects/b83750567fd84b7785ddc8db9f545331/users/ad2802f4404046df8daaf360777f7227/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:21.960091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:21.960413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:21.960696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.004239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] MFA Rules not processed for user `ad2802f4404046df8daaf360777f7227`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:22.008541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:22.100152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1285/2565] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:22.101337 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.182519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853435a-ce5a-445a-a45d-ebad9a2e4668 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.187592 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.188383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.188733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.189111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.189885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.191884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.272077 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1281/2566] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:22 2022] POST /identity/v3/projects => generated 407 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:22.276229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.289003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f0e3-ee4e-4cb7-a383-7eac6bf7dc29 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.351238 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.351909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.352197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.352475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.353204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.355116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.441662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1286/2567] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:22 2022] POST /identity/v3/users => generated 310 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:22.446441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.457059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de39d35-a196-4e2a-a97f-33a45dea7936 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.462315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.473249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.473249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.473249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.473249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.473249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.479672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1282/2568] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:22.483832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.495588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039e870-ae07-46c3-af13-1029d793aad2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.500849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.501763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.502045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.502314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] PATH_INFO: `/v3/projects/c556f4036bed40b2a82e774a4893e023/users/df55d1690c4949e28ece1c76885a76a5/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.518396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c556f4036bed40b2a82e774a4893e023, user_id=df55d1690c4949e28ece1c76885a76a5, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.522104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.618903 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1287/2569] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:22 2022] PUT /identity/v3/projects/c556f4036bed40b2a82e774a4893e023/users/df55d1690c4949e28ece1c76885a76a5/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:22.623223 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.633798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c80995-0d20-4f65-8971-417053085352 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.714030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.714806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.715143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.715422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.716213 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.718041 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f45f00-d706-4f68-9b67-b836ddc56704 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.731977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1283/2570] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:22.733294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.744523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565516-1a46-44c0-a29e-0f871702ab89 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.758830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.759744 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.760020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.760288 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] PATH_INFO: `/v3/projects/c556f4036bed40b2a82e774a4893e023/users/df55d1690c4949e28ece1c76885a76a5/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.762526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c556f4036bed40b2a82e774a4893e023, user_id=df55d1690c4949e28ece1c76885a76a5, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.774484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:22.786761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1288/2571] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:22 2022] PUT /identity/v3/projects/c556f4036bed40b2a82e774a4893e023/users/df55d1690c4949e28ece1c76885a76a5/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:22.797386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.797604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.797799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.828972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] MFA Rules not processed for user `df55d1690c4949e28ece1c76885a76a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:22.830264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:22.874224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:22.904402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1284/2572] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:22 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:22.915585 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:22.917402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:22.928210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:22.976771 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94f32-0aae-4d99-9912-3c38d5fecf36 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:22.980403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:22.981109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:22.981395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:22.981671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:22.991347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:22.992764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:23.002163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1289/2573] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:22 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:23.533320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:23.543425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830c7d1-9164-4739-8cd9-20c19622a294 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:23.547273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:23.557680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:23.557680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:23.557680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:23.608524 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:23.610049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:23.619847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1285/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:32.935607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:32.939979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:32.995116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49308e9f-e23d-4f05-bd55-c04d978ab2d4 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:32.998951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:32.999592 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:32.999873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:33.000150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:33.001944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:33.064898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:33.066353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213138ee-f4ea-4c0c-bfff-8a6113e86f40 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:33.076901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1290/2575] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:33.615768 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=39 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:34.586701 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:34.598481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688ec29-176a-411d-af44-1a7b25d47313 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:34.603320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:34.604258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:34.604615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:34.604985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] PATH_INFO: `/v3/users/ad2802f4404046df8daaf360777f7227` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:34.606265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2802f4404046df8daaf360777f7227)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:34.608341 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:34.851116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2802f4404046df8daaf360777f7227'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:34.868734 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2802f4404046df8daaf360777f7227'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:34.869851 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2802f4404046df8daaf360777f7227'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:34.870683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2802f4404046df8daaf360777f7227'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:34.878962 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2802f4404046df8daaf360777f7227'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:34.997442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1286/2576] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:34 2022] DELETE /identity/v3/users/ad2802f4404046df8daaf360777f7227 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:35.020711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:35.020711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:35.020711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:35.033497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:35.034932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:35.098036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1291/2577] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:35.108608 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:35.110275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:35.181811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe83b1d-616c-4e4b-8473-faf696af9717 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:35.188254 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:35.188921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:35.189278 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:35.189558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:35.199510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:35.201395 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:35.212434 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1287/2578] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:35.718136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:35.721369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:35.735141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:35.751176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca65957-475b-431b-8774-143c695e1ea4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:35.756101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:35.763731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:35.763731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:35.763731 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] PATH_INFO: `/v3/projects/b83750567fd84b7785ddc8db9f545331` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:35.764200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83750567fd84b7785ddc8db9f545331)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:35.766342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:35.896875 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83750567fd84b7785ddc8db9f545331', 'request_id': 'req-524493bf-5d32-41a7-84f1-2c2971518750'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:35.903622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83750567fd84b7785ddc8db9f545331', 'request_id': 'req-524493bf-5d32-41a7-84f1-2c2971518750'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:35.983340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83750567fd84b7785ddc8db9f545331', 'request_id': 'req-524493bf-5d32-41a7-84f1-2c2971518750'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:35.995473 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] The token cache is being invalidate because project b83750567fd84b7785ddc8db9f545331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:35.995874 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:35.996532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:35.998313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1292/2579] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:35 2022] DELETE /identity/v3/projects/b83750567fd84b7785ddc8db9f545331 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:36.002181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:36.012925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06331987-016f-426d-9c7a-793b29c912f3 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:36.085667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:36.086560 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:36.086845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:36.087126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] PATH_INFO: `/v3/users/df55d1690c4949e28ece1c76885a76a5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:36.088335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=df55d1690c4949e28ece1c76885a76a5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:36.090324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:36.111462 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:36.302860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df55d1690c4949e28ece1c76885a76a5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:36.324328 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df55d1690c4949e28ece1c76885a76a5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:36.325139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df55d1690c4949e28ece1c76885a76a5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:36.325779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df55d1690c4949e28ece1c76885a76a5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:36.334106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df55d1690c4949e28ece1c76885a76a5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:36.426813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1288/2580] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:36 2022] DELETE /identity/v3/users/df55d1690c4949e28ece1c76885a76a5 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:37.126203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:37.127905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:37.145104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524493bf-5d32-41a7-84f1-2c2971518750 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:37.227174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:37.228240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.228588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.228933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] PATH_INFO: `/v3/projects/c556f4036bed40b2a82e774a4893e023` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.230232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c556f4036bed40b2a82e774a4893e023)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:37.232143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:37.280865 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:37.283618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:37.303620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff377db-0321-4ec7-93ac-30d80165eafc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:37.309869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:37.313528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.313528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.313528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] PATH_INFO: `/v3/users/628e662a3b49438f9930a2d160658b7c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.327127 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628e662a3b49438f9930a2d160658b7c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:37.329190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:37.356992 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c556f4036bed40b2a82e774a4893e023', 'request_id': 'req-4f40eb11-5c0a-4196-b830-31ae500a908f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.366714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c556f4036bed40b2a82e774a4893e023', 'request_id': 'req-4f40eb11-5c0a-4196-b830-31ae500a908f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.482160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c556f4036bed40b2a82e774a4893e023', 'request_id': 'req-4f40eb11-5c0a-4196-b830-31ae500a908f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.499457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] The token cache is being invalidate because project c556f4036bed40b2a82e774a4893e023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:37.500132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.500860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f40eb11-5c0a-4196-b830-31ae500a908f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.502769 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1293/2581] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:37 2022] DELETE /identity/v3/projects/c556f4036bed40b2a82e774a4893e023 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:37.509394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a36cc-b74d-4d32-b663-66a4d2796669 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.509883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a36cc-b74d-4d32-b663-66a4d2796669 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.510261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a36cc-b74d-4d32-b663-66a4d2796669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.535140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-269a36cc-b74d-4d32-b663-66a4d2796669 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:37.537157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-269a36cc-b74d-4d32-b663-66a4d2796669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:37.547445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:37.611480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1294/2582] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:37.620613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.620906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.621239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.640845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628e662a3b49438f9930a2d160658b7c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.647810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:37.648474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:37.662153 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1295/2583] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:37.663175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:37.674140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722aee-8289-45fd-8fda-f3cd142ae092 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:37.683796 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:37.684570 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.685648 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.685932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.689983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:37.689983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:37.690214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628e662a3b49438f9930a2d160658b7c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.690491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:37.690778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628e662a3b49438f9930a2d160658b7c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.691520 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628e662a3b49438f9930a2d160658b7c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.708776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628e662a3b49438f9930a2d160658b7c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:37.712784 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1296/2584] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:37.716696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:37.718569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:37.736906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c0cad-e802-4a2c-8357-a232204aceb2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:37.742141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:37.742738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:37.743055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:37.743355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:37.743984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:37.745947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:37.828889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1289/2585] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:37 2022] DELETE /identity/v3/users/628e662a3b49438f9930a2d160658b7c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:37.864376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1297/2586] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:37 2022] POST /identity/v3/projects => generated 399 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:37.865997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:37.932143 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:37.938023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe6249-4c36-487f-b7ed-e84f231e8ce5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.004874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.005994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.006443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.006767 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.007629 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.009606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.116764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1290/2587] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:37 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:38.118050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.128373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae35c-9624-41f8-bf69-1f19a85d24ac admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.133793 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.137016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.137444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.137880 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.138878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.141932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.143120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.146421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:38.156358 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1298/2588] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:38.157633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.169898 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06100101-ee41-48be-bd4b-fb65ad266e88 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.178688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de17e9e-8134-4007-b0cd-0d4197c63158 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.187889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.188581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.188863 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.189251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.189425 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.189904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.191175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.192441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.192606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.192958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] PATH_INFO: `/v3/projects/a3bb5db6207441a1b5030af42f291666` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.208246 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bb5db6207441a1b5030af42f291666)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.210311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.237618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1299/2589] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:38.237618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.245029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7523-1590-4f13-81b6-e24ce466826f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.249659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.251621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.251916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.252195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] PATH_INFO: `/v3/projects/71fee3f1146746f9b8b63eac8f0b8fe6/users/eaa5465c943645f2b8d1c33386c13b1c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.269214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fee3f1146746f9b8b63eac8f0b8fe6, user_id=eaa5465c943645f2b8d1c33386c13b1c, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.272063 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a5e9f-2e0f-4331-b478-79d58848f9e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.328316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1300/2590] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:38 2022] PUT /identity/v3/projects/71fee3f1146746f9b8b63eac8f0b8fe6/users/eaa5465c943645f2b8d1c33386c13b1c/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:38.339249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.339557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.339840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.339956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bb5db6207441a1b5030af42f291666', 'request_id': 'req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:38.346384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bb5db6207441a1b5030af42f291666', 'request_id': 'req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:38.369343 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] MFA Rules not processed for user `eaa5465c943645f2b8d1c33386c13b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:38.370639 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:38.400783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bb5db6207441a1b5030af42f291666', 'request_id': 'req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:38.418236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] The token cache is being invalidate because project a3bb5db6207441a1b5030af42f291666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:38.418822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:38.421392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f9c0e899-b4e8-456e-b2a9-adb5a219db5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:38.424093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1291/2591] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:38 2022] DELETE /identity/v3/projects/a3bb5db6207441a1b5030af42f291666 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:38.430386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933199-5b71-4e80-addd-7a42fd05ee6d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.433121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933199-5b71-4e80-addd-7a42fd05ee6d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.433644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933199-5b71-4e80-addd-7a42fd05ee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.474370 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-63933199-5b71-4e80-addd-7a42fd05ee6d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:38.475761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-63933199-5b71-4e80-addd-7a42fd05ee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:38.489323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1301/2592] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:38.501224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.502835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:38.576499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1292/2593] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:38.584013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cd114-535f-45c3-bd35-0abef3284d7f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.589177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.589679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.590142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.597931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.597931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.597931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.597931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.606522 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.608154 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.618870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1302/2594] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:38.621070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:38.622574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:38.632108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1293/2595] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:38.636723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.650547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f3d3-bc86-46ec-a5fd-52d28bfafb46 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.656042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.656711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.657160 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.657547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.668386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:38.668386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.668386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.677080 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1303/2596] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:38.678847 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.689960 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7b536-a6a9-49df-9a47-0a258d0b3b91 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.696844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.697672 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.698039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.698381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.699162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.701155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.709912 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:38.808794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1294/2597] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:38.813140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:38.824207 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70937-6a7d-4f1e-8d8e-d9923b53e8b3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:38.887131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:38.887814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:38.888103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:38.888376 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:38.889083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:38.891004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:38.998226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1304/2598] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:38 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:38.999543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:39.010752 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a22c2a-79fd-4f53-97b0-5e0173bd9930 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:39.015481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:39.016140 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:39.016423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:39.016700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:39.017463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:39.019037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:39.030604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1295/2599] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:39.035013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:39.045412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f866-0527-4b5c-b6f5-4d48397179f2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:39.049934 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:39.050580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:39.050862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:39.051741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:39.052453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:39.054062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12eb9-43ba-4b7a-ba25-fdd636b6abe0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:39.064757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1305/2600] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:39.069754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:39.079736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae31f8-23bf-4b60-866f-176d1c5d0a59 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:39.084422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:39.085343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:39.085989 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:39.086468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] PATH_INFO: `/v3/projects/44bc670e6a714cbc88a2a41990cec6ad/users/cdedf8957dd84ad3a2ca113a31b4d6d3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:39.088406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:39.103014 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bc670e6a714cbc88a2a41990cec6ad, user_id=cdedf8957dd84ad3a2ca113a31b4d6d3, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:39.105798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:39.204650 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1296/2601] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:39 2022] PUT /identity/v3/projects/44bc670e6a714cbc88a2a41990cec6ad/users/cdedf8957dd84ad3a2ca113a31b4d6d3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:39.215652 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:39.216140 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:39.216240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:39.244175 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] MFA Rules not processed for user `cdedf8957dd84ad3a2ca113a31b4d6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:39.245516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:39.309778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1306/2602] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:39.321426 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:39.323114 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:39.396197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346f348-2b75-42d2-ab51-8ed05e355dc2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:39.402046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:39.402696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:39.403190 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:39.403291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:39.411708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:39.413310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed510ff-3dc9-4f36-a934-92ccd1587573 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:39.422178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1297/2603] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:45.817400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:45.827296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:45.873256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688443e6-39c4-43ee-92fe-b7cab0deb2a8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:45.877686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:45.878546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:45.878828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:45.879117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] PATH_INFO: `/v3/users/cdedf8957dd84ad3a2ca113a31b4d6d3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:45.880438 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdedf8957dd84ad3a2ca113a31b4d6d3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:45.882394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:46.069204 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdedf8957dd84ad3a2ca113a31b4d6d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:46.075957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdedf8957dd84ad3a2ca113a31b4d6d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:46.088492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdedf8957dd84ad3a2ca113a31b4d6d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:46.089198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdedf8957dd84ad3a2ca113a31b4d6d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:46.180104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdedf8957dd84ad3a2ca113a31b4d6d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:46.186216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1307/2604] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:45 2022] DELETE /identity/v3/users/cdedf8957dd84ad3a2ca113a31b4d6d3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:46.190544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:46.191006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:46.191168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:46.209797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:46.210988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:46.289242 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1298/2605] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:46.297366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:46.298815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:46.329774 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:46.331299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:46.366240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca316f-f858-4428-b73b-8b8fa245f649 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:46.368055 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:46.370758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:46.371447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:46.371746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:46.372003 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:46.386675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:46.388121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:46.398338 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1308/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:46.401399 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4d5da-8faa-4361-897d-4ae07d249729 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:46.405109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:46.405832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:46.406185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:46.406596 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:46.408446 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:46.421876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:46.423315 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03e1c4-ca0d-49f4-a627-4037c0087bce service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:46.434428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1299/2607] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:47.076190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.081161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:47.093415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880868-0682-42d3-8496-dceba7884488 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.099142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.100019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.100305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.100584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] PATH_INFO: `/v3/projects/44bc670e6a714cbc88a2a41990cec6ad` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.102333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bc670e6a714cbc88a2a41990cec6ad)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.104544 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.199299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:47.211059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bc670e6a714cbc88a2a41990cec6ad', 'request_id': 'req-b48c15db-dae0-42e2-9895-cae6e87f9c76'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:47.217312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bc670e6a714cbc88a2a41990cec6ad', 'request_id': 'req-b48c15db-dae0-42e2-9895-cae6e87f9c76'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:47.303450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bc670e6a714cbc88a2a41990cec6ad', 'request_id': 'req-b48c15db-dae0-42e2-9895-cae6e87f9c76'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:47.314978 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] The token cache is being invalidate because project 44bc670e6a714cbc88a2a41990cec6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:47.315262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:47.315899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b48c15db-dae0-42e2-9895-cae6e87f9c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:47.318690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1309/2608] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:47 2022] DELETE /identity/v3/projects/44bc670e6a714cbc88a2a41990cec6ad => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:47.323635 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.324132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.324245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.342259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:47.343502 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:47.405756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1300/2609] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:47.414404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.415245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.415245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.435862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:47.437319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:47.451983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1310/2610] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:47.454178 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.466569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4eb5d-2133-446d-bb8f-8dc8cf0f6cd3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.471936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.472510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.478036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.478036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.478036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:47.478036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.478036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.497002 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1301/2611] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:47.498043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.509152 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a2ef0-bbfc-41ed-9bfc-5d61d44b9b9e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.517449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.518193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.518476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.518746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.519500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.522138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.646098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1311/2612] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:47 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:47.650481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.662844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d99-347e-44e8-a133-6ee15c81efad admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.733715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.734488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.734777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.735052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.735823 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.737846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.810191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1302/2613] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:47 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:47.811388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.822821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e886c5-8d1d-485f-8722-612bf321b595 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.827644 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.828331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.828657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.828933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.829713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.831465 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.843734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1312/2614] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:47.848402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.858628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5e7c-4b7e-4736-b0a1-c2962823c944 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.863493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.864097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.864379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.864658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.865414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.867000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd6ab5-7f0a-4e05-a3d1-804b43925cfa admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:47.880285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1303/2615] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:47.881667 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:47.893232 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4a896-c773-4fad-9c49-d455ed772c3d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:47.898668 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:47.899645 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:47.899902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:47.900177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] PATH_INFO: `/v3/projects/c287164ffa9e49b7a71e40020361d817/users/b0c46f675eb74cac8e7601ab7b55452f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:47.916073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c287164ffa9e49b7a71e40020361d817, user_id=b0c46f675eb74cac8e7601ab7b55452f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:47.919106 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.029697 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1313/2616] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:47 2022] PUT /identity/v3/projects/c287164ffa9e49b7a71e40020361d817/users/b0c46f675eb74cac8e7601ab7b55452f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:48.042506 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.042797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.043080 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.070422 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] MFA Rules not processed for user `b0c46f675eb74cac8e7601ab7b55452f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:48.071784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:48.100128 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:48.140882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1304/2617] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:48.146628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.225010 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477b61-ef38-4876-8efd-89d4afde5c94 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.230216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.230878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.231156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.232015 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.232908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.234815 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.355349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1314/2618] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:48 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:48.357677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.369509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c46ee2-2871-42c8-8310-d83257fb623f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.435664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.436330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.436615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.436883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.437655 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.439565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.520247 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1305/2619] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:48 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:48.524122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.534786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8512cab-946b-4b46-851d-e4095205afb7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.540718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.541469 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.541800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.542073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.542801 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.544415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.562075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1315/2620] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:48.563249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.577431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587c48a-6c7e-4e70-8a30-f15a20f4c84e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.583191 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.585150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.585582 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.585953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] PATH_INFO: `/v3/projects/a664f73a2eca42348e38aaba8c60f816/users/904a2841a66a4487a0ce5d8d45efc155/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.606493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a664f73a2eca42348e38aaba8c60f816, user_id=904a2841a66a4487a0ce5d8d45efc155, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.610417 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.697222 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1306/2621] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:48 2022] PUT /identity/v3/projects/a664f73a2eca42348e38aaba8c60f816/users/904a2841a66a4487a0ce5d8d45efc155/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:48.698422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.711297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b589bd4-a55f-4727-8286-152614ce6f04 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.712922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.718726 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:48.762531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:48.810715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.811571 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.811931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.812310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.814006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.816205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.818871 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca63c-5ef7-4e06-895f-fcf4c42328a7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.826360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.827517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.827988 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.828269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] PATH_INFO: `/v3/users/eaa5465c943645f2b8d1c33386c13b1c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.829405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1316/2622] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:48.835944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:48.836107 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa5465c943645f2b8d1c33386c13b1c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.836478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.856464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98d4-7250-4414-a7ee-c18eaac6833a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:48.869356 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:48.870457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.870803 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.871142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] PATH_INFO: `/v3/projects/a664f73a2eca42348e38aaba8c60f816/users/904a2841a66a4487a0ce5d8d45efc155/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.873656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a664f73a2eca42348e38aaba8c60f816, user_id=904a2841a66a4487a0ce5d8d45efc155, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:48.876412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e7d5c-3b7e-4c19-8aae-88116b1aa3e3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:48.888824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1317/2623] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:48 2022] PUT /identity/v3/projects/a664f73a2eca42348e38aaba8c60f816/users/904a2841a66a4487a0ce5d8d45efc155/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:48.900078 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107330a1-06f8-491f-a573-a2e936e571be None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:48.900366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107330a1-06f8-491f-a573-a2e936e571be None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:48.900704 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107330a1-06f8-491f-a573-a2e936e571be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:48.931331 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-107330a1-06f8-491f-a573-a2e936e571be None None] MFA Rules not processed for user `904a2841a66a4487a0ce5d8d45efc155`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:48.932611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-107330a1-06f8-491f-a573-a2e936e571be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:49.008890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1318/2624] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:49.020764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107330a1-06f8-491f-a573-a2e936e571be None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:49.022220 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-107330a1-06f8-491f-a573-a2e936e571be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:49.059100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:49.069634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa5465c943645f2b8d1c33386c13b1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:49.083010 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa5465c943645f2b8d1c33386c13b1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:49.083920 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa5465c943645f2b8d1c33386c13b1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:49.084101 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107330a1-06f8-491f-a573-a2e936e571be None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:49.084721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa5465c943645f2b8d1c33386c13b1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:49.087760 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:49.088408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:49.088691 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:49.088969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:49.090830 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:49.095895 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa5465c943645f2b8d1c33386c13b1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:49.104750 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:49.106299 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413cdd9-2934-4e04-8dc5-0fbb672d5bbb service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:49.116498 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1319/2625] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:49.179153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1307/2626] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:48 2022] DELETE /identity/v3/users/eaa5465c943645f2b8d1c33386c13b1c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:49.183810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:49.184244 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:49.184385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:49.201843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:49.208029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:49.270404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:49.305349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1320/2627] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:49.315252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:49.316840 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:49.396573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5143-2181-4e98-b1c4-0404c136e5d8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:49.400666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:49.401528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:49.401942 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:49.402316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:49.413690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:49.415295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:49.427012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1308/2628] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:50.005990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:50.007447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:50.019233 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c96c0b-ef37-409a-9711-dc2209b96cb4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:50.023848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:50.024696 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:50.024974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:50.025276 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] PATH_INFO: `/v3/projects/71fee3f1146746f9b8b63eac8f0b8fe6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:50.026464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fee3f1146746f9b8b63eac8f0b8fe6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:50.028362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:50.206053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fee3f1146746f9b8b63eac8f0b8fe6', 'request_id': 'req-26e83b4e-1088-4bb2-b897-6a2a2ab00086'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.214565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fee3f1146746f9b8b63eac8f0b8fe6', 'request_id': 'req-26e83b4e-1088-4bb2-b897-6a2a2ab00086'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.292087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fee3f1146746f9b8b63eac8f0b8fe6', 'request_id': 'req-26e83b4e-1088-4bb2-b897-6a2a2ab00086'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.308344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] The token cache is being invalidate because project 71fee3f1146746f9b8b63eac8f0b8fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:50.308655 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.309362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.311167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1321/2629] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:50 2022] DELETE /identity/v3/projects/71fee3f1146746f9b8b63eac8f0b8fe6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:50.601500 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:50.603039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:50.616125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fc90-a594-46a4-bdb8-2fff426c2e91 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:50.684287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:50.685349 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:50.685682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:50.685973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] PATH_INFO: `/v3/users/b0c46f675eb74cac8e7601ab7b55452f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:50.687424 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c46f675eb74cac8e7601ab7b55452f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:50.689406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:50.847130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:50.849521 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:50.864727 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83b4e-1088-4bb2-b897-6a2a2ab00086 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:50.866620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c46f675eb74cac8e7601ab7b55452f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.869896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:50.870774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:50.871059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:50.871454 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] PATH_INFO: `/v3/users/3e12d902f94b40cea088dc6f24dcb444` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:50.874281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:50.881475 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c46f675eb74cac8e7601ab7b55452f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.882194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c46f675eb74cac8e7601ab7b55452f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.882848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c46f675eb74cac8e7601ab7b55452f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.886464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e12d902f94b40cea088dc6f24dcb444)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:50.888418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:50.889820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7f415017-7a32-4b16-aedb-d68f78eeaad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c46f675eb74cac8e7601ab7b55452f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:50.954343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1309/2630] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:50 2022] DELETE /identity/v3/users/b0c46f675eb74cac8e7601ab7b55452f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:50.959202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:50.959514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:50.959834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:50.977813 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:50.979001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:51.039229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1310/2631] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:51.049305 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:51.050668 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:51.055612 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e12d902f94b40cea088dc6f24dcb444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.063527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e12d902f94b40cea088dc6f24dcb444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.080758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e12d902f94b40cea088dc6f24dcb444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.081509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e12d902f94b40cea088dc6f24dcb444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.116071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1a9e6-7778-463a-a587-f46764343fa4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:51.119710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:51.120351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:51.120634 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:51.120909 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:51.129951 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:51.131372 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:51.140673 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1311/2632] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:51.160208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e12d902f94b40cea088dc6f24dcb444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.167334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1322/2633] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:50 2022] DELETE /identity/v3/users/3e12d902f94b40cea088dc6f24dcb444 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:51.545197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:51.545197 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:51.575905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623d66-f382-41f0-9470-bd7186fdacd0 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:51.632819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:51.632819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:51.634168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:51.635090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:51.635366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:51.635677 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] PATH_INFO: `/v3/projects/f762c3c164904a3bb6e64985744c1765` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:51.647792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f762c3c164904a3bb6e64985744c1765)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:51.649721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:51.654545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b500-d3bf-40da-bd4e-94258dcac88d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:51.661925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:51.662065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:51.662976 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:51.663430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:51.663868 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] PATH_INFO: `/v3/projects/c287164ffa9e49b7a71e40020361d817` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:51.665364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c287164ffa9e49b7a71e40020361d817)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:51.667335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:51.692367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:51.817618 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f762c3c164904a3bb6e64985744c1765', 'request_id': 'req-167b0cf1-ef8e-4d17-927c-87a05535cb19'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.824311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f762c3c164904a3bb6e64985744c1765', 'request_id': 'req-167b0cf1-ef8e-4d17-927c-87a05535cb19'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.828805 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c287164ffa9e49b7a71e40020361d817', 'request_id': 'req-13b4faa3-c9f1-4683-a6a9-b85493557f04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.835672 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c287164ffa9e49b7a71e40020361d817', 'request_id': 'req-13b4faa3-c9f1-4683-a6a9-b85493557f04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.902877 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f762c3c164904a3bb6e64985744c1765', 'request_id': 'req-167b0cf1-ef8e-4d17-927c-87a05535cb19'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.914681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] The token cache is being invalidate because project f762c3c164904a3bb6e64985744c1765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:51.914963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.915595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-167b0cf1-ef8e-4d17-927c-87a05535cb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.917227 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1312/2634] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:51 2022] DELETE /identity/v3/projects/f762c3c164904a3bb6e64985744c1765 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:51.921993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356c2-6615-42df-baf8-a01d018e4ea4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:51.922287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356c2-6615-42df-baf8-a01d018e4ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:51.922567 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356c2-6615-42df-baf8-a01d018e4ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:51.940123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a33356c2-6615-42df-baf8-a01d018e4ea4 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:51.941356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a33356c2-6615-42df-baf8-a01d018e4ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:51.971596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c287164ffa9e49b7a71e40020361d817', 'request_id': 'req-13b4faa3-c9f1-4683-a6a9-b85493557f04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.982779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] The token cache is being invalidate because project c287164ffa9e49b7a71e40020361d817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:51.983165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.983879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:51.985687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1323/2635] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:51 2022] DELETE /identity/v3/projects/c287164ffa9e49b7a71e40020361d817 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:51.989400 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:51.991065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:52.003467 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1313/2636] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:52.012237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.012546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.012827 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.031835 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:52.033088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:52.041541 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1314/2637] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:52.045769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.055138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27080-8b6b-4984-bc5a-ad7e14ff52f2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.056393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4faa3-c9f1-4683-a6a9-b85493557f04 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.060613 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.061294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.061580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.061857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.062583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:52.062889 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.064490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.068298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.069197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.069475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.069746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] PATH_INFO: `/v3/users/904a2841a66a4487a0ce5d8d45efc155` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.070946 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=904a2841a66a4487a0ce5d8d45efc155)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.072881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.076972 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1315/2638] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:59:52.081496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.091892 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e5ff2-7d76-4675-85e3-9b35ebb6963e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.096685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.097364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.097651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.097930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.098636 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.100817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.185945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1316/2639] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:52.190134 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.247906 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:52.248268 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '904a2841a66a4487a0ce5d8d45efc155'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.251661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7b0b7-641e-4d32-92ad-d31c1a314457 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.255442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '904a2841a66a4487a0ce5d8d45efc155'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.256937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.257640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.257924 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.258204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.258899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.260821 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.266869 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '904a2841a66a4487a0ce5d8d45efc155'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.267701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '904a2841a66a4487a0ce5d8d45efc155'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.321159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1317/2640] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:52.325367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.331599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '904a2841a66a4487a0ce5d8d45efc155'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.335156 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300d360-9c57-4ebc-aaf8-40e95aa25378 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.338776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1324/2641] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:51 2022] DELETE /identity/v3/users/904a2841a66a4487a0ce5d8d45efc155 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:52.340057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.344049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.344049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.344049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.344049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.344049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.354303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1318/2642] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:52.357920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.359657 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:52.395861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:52.429957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48343afd-5bbf-4d06-be07-cbabfdf87e23 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.434719 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.435449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.435735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.436013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.436739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.438387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab380-f588-40a0-8e70-3614443ad5ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.449736 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1325/2643] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:52.454030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.464863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616f60-c8a5-40e9-a764-c93a70f9c4d4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.469725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.470601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.470888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.471166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] PATH_INFO: `/v3/projects/f74dc4d8393e482b8b180b33ba7de582/users/e87b609824ac4c6685e218e2d5a009b5/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.487457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74dc4d8393e482b8b180b33ba7de582, user_id=e87b609824ac4c6685e218e2d5a009b5, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.490790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.560990 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1319/2644] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:52 2022] PUT /identity/v3/projects/f74dc4d8393e482b8b180b33ba7de582/users/e87b609824ac4c6685e218e2d5a009b5/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:52.573355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.573829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.574121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.603716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] MFA Rules not processed for user `e87b609824ac4c6685e218e2d5a009b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:52.604932 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:52.671282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1326/2645] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:52.675758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.733841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.735246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:52.742837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd97e-081b-45d8-9d80-66b95d9c8439 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.749708 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fa49e-6647-4a54-b916-73d0553dad2f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.755052 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.756264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.756521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.756712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.757017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] PATH_INFO: `/v3/projects/a664f73a2eca42348e38aaba8c60f816` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.757225 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.758202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.758457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a664f73a2eca42348e38aaba8c60f816)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.758983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.759820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.760846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.761905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.851574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1320/2646] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:52.855494 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:52.881560 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a664f73a2eca42348e38aaba8c60f816', 'request_id': 'req-af4fb710-64bf-4f4b-957a-7d982a0f5726'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.887909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a664f73a2eca42348e38aaba8c60f816', 'request_id': 'req-af4fb710-64bf-4f4b-957a-7d982a0f5726'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.917820 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05c860-2030-4c72-ba5c-1543f87ee4b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:52.958455 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:52.960124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a664f73a2eca42348e38aaba8c60f816', 'request_id': 'req-af4fb710-64bf-4f4b-957a-7d982a0f5726'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.971256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] The token cache is being invalidate because project a664f73a2eca42348e38aaba8c60f816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:52.971551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.972298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-af4fb710-64bf-4f4b-957a-7d982a0f5726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:52.972824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:52.973540 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.974335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.974526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1327/2647] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:52 2022] DELETE /identity/v3/projects/a664f73a2eca42348e38aaba8c60f816 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:52.974788 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.975661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:52.977654 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:52.979505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb4801-87d7-4dca-bed4-a60069916f78 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:52.979839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb4801-87d7-4dca-bed4-a60069916f78 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:52.980124 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb4801-87d7-4dca-bed4-a60069916f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:52.998973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3dfb4801-87d7-4dca-bed4-a60069916f78 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:53.000181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3dfb4801-87d7-4dca-bed4-a60069916f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.008367 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1328/2648] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:53.016088 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73768406-488c-4f76-a96d-bf367b36189d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.016382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73768406-488c-4f76-a96d-bf367b36189d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.016665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73768406-488c-4f76-a96d-bf367b36189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.034907 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-73768406-488c-4f76-a96d-bf367b36189d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:53.036263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73768406-488c-4f76-a96d-bf367b36189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.044778 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1329/2649] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:53.048732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73768406-488c-4f76-a96d-bf367b36189d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.056554 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1321/2650] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:52 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:53.058155 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73768406-488c-4f76-a96d-bf367b36189d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.061678 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.062818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.063273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.063458 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.063743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.064016 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.064730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:53.065167 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.066674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.075087 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a6ba-a6c0-44f2-8b5d-9ce7989b6591 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.078715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1330/2651] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 12:59:53.079925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.080588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.080874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.081245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.082019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.082741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.084100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.093395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125307a9-bc96-467b-acbf-5a179228714c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.097765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1322/2652] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:53.099343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.101695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.102342 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.102624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.102899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.103586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.105593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.109639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf59c-73bb-4644-ba4b-9ff4c946423f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.114437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.115384 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.115526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:53.115667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.115965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] PATH_INFO: `/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef/users/8157407d28464e419358ee8d2e59c216/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.131037 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd4eaa308a54662a3d5365b4271b1ef, user_id=8157407d28464e419358ee8d2e59c216, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.133805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.195984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1331/2653] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:53.200036 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.217472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1323/2654] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:53 2022] PUT /identity/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef/users/8157407d28464e419358ee8d2e59c216/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:53.221423 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.261816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2a4c-e9e0-41b7-9709-cb4edb90bfa5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.284808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb02c-f7a4-4717-9e80-03f90760b68c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.290886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.291525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.291814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.292090 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.292822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.294445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.305440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1324/2655] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:53.309336 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.318944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175223d7-8b06-4d42-ac3d-a53123f52659 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.320224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.320923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.321235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.321508 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.322188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.323699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.324082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.324553 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.324842 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.325149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] PATH_INFO: `/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef/users/8157407d28464e419358ee8d2e59c216/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.327149 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd4eaa308a54662a3d5365b4271b1ef, user_id=8157407d28464e419358ee8d2e59c216, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.329797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684103ea-37ab-415d-b7eb-e09f8215d190 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.341150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1325/2656] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:53 2022] PUT /identity/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef/users/8157407d28464e419358ee8d2e59c216/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:53.351721 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.352011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.352292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.378209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] MFA Rules not processed for user `8157407d28464e419358ee8d2e59c216`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:53.379414 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.394941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1332/2657] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 12:59:53.399811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.409870 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5749e-960a-4458-a5b6-5365079ad922 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.449876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1326/2658] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:53.462297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.463652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.467339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.468006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.468293 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.468565 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.469314 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.470900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.472879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:53.480983 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1333/2659] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:53.484694 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.494518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572de1-dc40-4b76-9e2d-80fd0cc55fad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.499000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.499960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.500249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.500518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.501253 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.502810 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.513081 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1334/2660] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 12:59:53.517229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.523159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca968c53-ecd6-430c-b900-af73680844e1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.527806 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ca6e0-ba26-4588-bfb8-bafdc219d6d3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.527955 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.528453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.528765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.529066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.532511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.533409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.533695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.536075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] PATH_INFO: `/v3/projects/1c11be853c664b43ae598948b5db7879/users/f631395917b542d09e9d68de8f91a5b3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.538513 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.539935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a85e99-f846-49bf-b43a-10f603d006c3 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.542952 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:53.549156 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c11be853c664b43ae598948b5db7879, user_id=f631395917b542d09e9d68de8f91a5b3, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.549565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1327/2661] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:53.553883 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.635281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1335/2662] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:53 2022] PUT /identity/v3/projects/1c11be853c664b43ae598948b5db7879/users/f631395917b542d09e9d68de8f91a5b3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:53.647257 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.647538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.647814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.678649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] MFA Rules not processed for user `f631395917b542d09e9d68de8f91a5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:53.680578 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.746638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1328/2663] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:53.751301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.779343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.825467 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668316a-22f7-4f69-a287-57aa372bb09c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.830500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.831274 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.831557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.831829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.832513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.834513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.838630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c87395-7c94-4784-a85a-bdc2a474b5e6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:53.842317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:53.842965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:53.843249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:53.843525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:53.845421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:53.902538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:53.906129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:53.907547 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:53.916706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1329/2664] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 12:59:53.949897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1336/2665] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:53.954527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:53.967103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b3ad4-8410-4657-b3ec-1114f983f8cd service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.022546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.023211 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.023492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.023764 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.024435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.026365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.119304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1330/2666] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:53 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:54.123393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:54.134728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ab24d-5b46-4d88-b923-3c2fe742f083 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.139362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.140060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.140347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.140621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.141398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.142984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.154059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1337/2667] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:54.158287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:54.168737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348519c7-fb7c-4062-aba0-7c90d3ed9f86 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.173359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.174239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.174516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.174783 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] PATH_INFO: `/v3/projects/425c582ae04c426c9e1b74fdb00f4d96/users/a8defa3759c447ffa2b82926a5c8277e/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.191509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=425c582ae04c426c9e1b74fdb00f4d96, user_id=a8defa3759c447ffa2b82926a5c8277e, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.194324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.268304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1331/2668] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:54 2022] PUT /identity/v3/projects/425c582ae04c426c9e1b74fdb00f4d96/users/a8defa3759c447ffa2b82926a5c8277e/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:54.272486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:54.282410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1336a3-9e0a-4feb-986b-26d0a489e3fc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.346902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.347580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.347864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.348132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.349430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.351193 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d17afb-6fc5-4aec-bc8b-ae234d1ff839 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.362389 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1338/2669] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:54.367450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:54.378110 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7beae6-f58c-4e39-a244-06199d15a94b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.383005 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.383890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.384175 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.387161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] PATH_INFO: `/v3/projects/425c582ae04c426c9e1b74fdb00f4d96/users/a8defa3759c447ffa2b82926a5c8277e/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.387161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425c582ae04c426c9e1b74fdb00f4d96, user_id=a8defa3759c447ffa2b82926a5c8277e, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.389165 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.399692 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1332/2670] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:54 2022] PUT /identity/v3/projects/425c582ae04c426c9e1b74fdb00f4d96/users/a8defa3759c447ffa2b82926a5c8277e/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:54.410146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.410523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.410809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.438920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] MFA Rules not processed for user `a8defa3759c447ffa2b82926a5c8277e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:54.440123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:54.484607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:54.508135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1339/2671] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:54.519054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:54.574326 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b2543-955a-4859-9ca0-7a219fcdc087 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:54.578012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:54.578646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:54.578927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:54.581385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:54.589006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:54.590532 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347cc85-ebb4-43ff-a8d8-c710014b015c service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:54.601387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1333/2672] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:55.095203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:55.155852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5835e-66d6-41da-881a-737de4a44a21 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:55.160313 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:55.161350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:55.161961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:55.162239 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] PATH_INFO: `/v3/users/f631395917b542d09e9d68de8f91a5b3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:55.163434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f631395917b542d09e9d68de8f91a5b3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:55.165392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:55.305395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f631395917b542d09e9d68de8f91a5b3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:55.312122 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f631395917b542d09e9d68de8f91a5b3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:55.324594 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f631395917b542d09e9d68de8f91a5b3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:55.325297 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f631395917b542d09e9d68de8f91a5b3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:55.381741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f631395917b542d09e9d68de8f91a5b3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:55.388172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1340/2673] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:55 2022] DELETE /identity/v3/users/f631395917b542d09e9d68de8f91a5b3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:55.392681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:55.393285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:55.393590 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:55.414486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:55.415691 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:55.460744 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:55.475878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1334/2674] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 12:59:55.485630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:55.487053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:55.547679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c765b8-caae-41fc-95db-019fe25707c9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:55.551536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:55.552747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:55.553027 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:55.553330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:55.562854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:55.564222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:55.574782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1341/2675] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 12:59:56.058591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:56.060104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:56.072606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7c754-e3c8-4efd-af31-7c4d0008df31 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:56.078407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:56.079552 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:56.082639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:56.082639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] PATH_INFO: `/v3/projects/1c11be853c664b43ae598948b5db7879` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:56.082639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c11be853c664b43ae598948b5db7879)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:56.084683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:56.217093 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c11be853c664b43ae598948b5db7879', 'request_id': 'req-202ce61e-3763-406f-9c47-c537b600c468'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.223387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c11be853c664b43ae598948b5db7879', 'request_id': 'req-202ce61e-3763-406f-9c47-c537b600c468'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.302757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c11be853c664b43ae598948b5db7879', 'request_id': 'req-202ce61e-3763-406f-9c47-c537b600c468'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.313893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] The token cache is being invalidate because project 1c11be853c664b43ae598948b5db7879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:56.314181 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.314808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.316350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1335/2676] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:56 2022] DELETE /identity/v3/projects/1c11be853c664b43ae598948b5db7879 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:56.320692 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:56.326683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:56.330340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede3dd-8129-4935-9037-a058c87dc576 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:56.387002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:56.387861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:56.388142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:56.388413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] PATH_INFO: `/v3/users/a8defa3759c447ffa2b82926a5c8277e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:56.389638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8defa3759c447ffa2b82926a5c8277e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:56.391551 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:56.550144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8defa3759c447ffa2b82926a5c8277e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.557210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8defa3759c447ffa2b82926a5c8277e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.569277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8defa3759c447ffa2b82926a5c8277e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.569940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8defa3759c447ffa2b82926a5c8277e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.639961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f76b2c2-bb3c-43e6-be81-7b9b49472bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8defa3759c447ffa2b82926a5c8277e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:56.646909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1342/2677] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 12:59:56 2022] DELETE /identity/v3/users/a8defa3759c447ffa2b82926a5c8277e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:57.064201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.065671 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:57.081452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ce61e-3763-406f-9c47-c537b600c468 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.140894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.141790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.142070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.142342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] PATH_INFO: `/v3/projects/425c582ae04c426c9e1b74fdb00f4d96` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.143510 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=425c582ae04c426c9e1b74fdb00f4d96)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.145468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.274664 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425c582ae04c426c9e1b74fdb00f4d96', 'request_id': 'req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:57.280739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425c582ae04c426c9e1b74fdb00f4d96', 'request_id': 'req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:57.352391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425c582ae04c426c9e1b74fdb00f4d96', 'request_id': 'req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:57.366307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] The token cache is being invalidate because project 425c582ae04c426c9e1b74fdb00f4d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 12:59:57.366946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:57.367905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8f88139d-4f1b-45cf-a5a6-61fecb5cfb4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 12:59:57.369662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1336/2678] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 12:59:57 2022] DELETE /identity/v3/projects/425c582ae04c426c9e1b74fdb00f4d96 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 12:59:57.374989 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c227-d592-4612-8c44-fea68285a879 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.375448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c227-d592-4612-8c44-fea68285a879 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.380587 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c227-d592-4612-8c44-fea68285a879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.393772 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c501c227-d592-4612-8c44-fea68285a879 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:57.394964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c501c227-d592-4612-8c44-fea68285a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:57.454619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1343/2679] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:57.462880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.463229 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.463521 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.470638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:57.484279 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:57.485516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:57.494574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1337/2680] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:57.498940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c227-d592-4612-8c44-fea68285a879 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.509002 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c227-d592-4612-8c44-fea68285a879 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.513723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.514362 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.520065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.520065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.520065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 12:59:57.520065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.520065 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.529391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1344/2681] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 12:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 12:59:57.533643 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.543735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca751b0-240c-41ca-a809-174308c7b0a7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.548725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.549543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.549831 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.550109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.550807 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.552745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.658266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1338/2682] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 12:59:57 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:57.663375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.673689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0988-b26d-4b12-abca-fa67d6d19345 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.682569 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 12:59:57.732577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.733191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.733474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.733743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.734415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.736312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.856312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1345/2683] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 12:59:57 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 12:59:57.860463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.870816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605b280-02fe-4c08-b571-a38cfcfc97b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.875556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.876210 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.878478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.878478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.878478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.879218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.890226 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1339/2684] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:57.893996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.904329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785b2e-3cd2-46cf-960d-30d09ff189ed admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.908927 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.909596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.909873 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.910144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.910853 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.912416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ac160-7bb4-41a4-b17e-cf4d4b7b2459 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:57.925843 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1346/2685] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 12:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 12:59:57.926680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:57.936860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fd574-fa7b-4928-b3cf-af17fc5cab12 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:57.941427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:57.942291 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:57.942572 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:57.942846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] PATH_INFO: `/v3/projects/a35eb5aec58b45fa9ae8da8d377103ce/users/2652c27445bf49a9bbd6b2148e8bfebb/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:57.957640 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35eb5aec58b45fa9ae8da8d377103ce, user_id=2652c27445bf49a9bbd6b2148e8bfebb, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:57.960787 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:58.023967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1340/2686] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 12:59:57 2022] PUT /identity/v3/projects/a35eb5aec58b45fa9ae8da8d377103ce/users/2652c27445bf49a9bbd6b2148e8bfebb/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 12:59:58.035014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:58.035318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:58.035619 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:58.061699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] MFA Rules not processed for user `2652c27445bf49a9bbd6b2148e8bfebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 12:59:58.062894 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:58.125749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1347/2687] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 12:59:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 12:59:58.137304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 12:59:58.138980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 12:59:58.197023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c4bad-b91e-47f2-a9d5-a2cfa48235ff admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 12:59:58.200945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 12:59:58.201619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 12:59:58.201901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 12:59:58.202174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 12:59:58.211303 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 12:59:58.213150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43894bc-fb4d-4fb6-9056-9085f2a3a6fe service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 12:59:58.222161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1341/2688] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 12:59:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:08.122864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:08.218142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:12.277164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:12.339294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331fb3c-3847-40a8-93b8-498f60cb7c1b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:12.343812 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:12.345842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:12.345842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:12.345842 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] PATH_INFO: `/v3/users/2652c27445bf49a9bbd6b2148e8bfebb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:12.346556 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2652c27445bf49a9bbd6b2148e8bfebb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:12.348447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:12.533031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2652c27445bf49a9bbd6b2148e8bfebb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:12.540514 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2652c27445bf49a9bbd6b2148e8bfebb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:12.552068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2652c27445bf49a9bbd6b2148e8bfebb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:12.552724 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2652c27445bf49a9bbd6b2148e8bfebb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:12.610943 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2652c27445bf49a9bbd6b2148e8bfebb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:12.617196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1348/2689] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:12 2022] DELETE /identity/v3/users/2652c27445bf49a9bbd6b2148e8bfebb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:12.621856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:12.622330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:12.622437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:12.639252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:12.640447 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:12.705619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1342/2690] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:12.715775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:12.717211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:12.776375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4989bd-fb48-493a-95cd-6fc2bcea222b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:12.780163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:12.780813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:12.781137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:12.781407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:12.791474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:12.795897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd45b85-512c-4d09-b8e6-b9686e278678 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:12.802399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1349/2691] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:13.230688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.232182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:13.245385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb68460-ab1b-4190-88e6-aed4c3413507 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.249991 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.255275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.255275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.255275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] PATH_INFO: `/v3/projects/a35eb5aec58b45fa9ae8da8d377103ce` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.255275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35eb5aec58b45fa9ae8da8d377103ce)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.255275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.366950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35eb5aec58b45fa9ae8da8d377103ce', 'request_id': 'req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:13.373292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35eb5aec58b45fa9ae8da8d377103ce', 'request_id': 'req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:13.451647 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35eb5aec58b45fa9ae8da8d377103ce', 'request_id': 'req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:13.463098 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] The token cache is being invalidate because project a35eb5aec58b45fa9ae8da8d377103ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:13.463386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:13.464013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3e942f7c-1c99-4fac-bc7b-e5550fba5cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:13.465589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1343/2692] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:13 2022] DELETE /identity/v3/projects/a35eb5aec58b45fa9ae8da8d377103ce => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:13.470545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.471048 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.471174 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.489022 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:13.490197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:13.509051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:13.549872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1350/2693] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:13.559310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.559597 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.559874 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.578845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:13.580070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:13.588111 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1344/2694] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:13.592420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.602518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c445-9527-431d-bb6d-cb347513e4c1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.607062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.607763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.608077 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.608324 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.609066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:00:13.609379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.610973 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.623879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1351/2695] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:00:13.628383 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.637965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031ae4-04f8-4bc9-adbd-ab1086dfed78 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.642731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.643369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.643652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.643921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.644606 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.646528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.743371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1345/2696] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:13 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:13.747642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.758098 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814f60-8a82-4d6d-ae46-cb1fe3eb7adb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.817004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.817699 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.817981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.818256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.819519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.821488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.924360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1352/2697] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:13 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:13.929193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.939790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77f3de-b11e-4ed1-90f2-92766bcf4cbd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.944516 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.945204 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.945485 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.945757 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.946474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.948124 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.958519 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1346/2698] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:13.962450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:13.972407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93bd0-8c4a-4711-9303-dd066138299d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:13.977926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:13.978578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:13.978867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:13.979145 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:13.979979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:13.981614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849dea2-fc0b-48e4-9ed0-6c1f85c728fe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:13.992541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1353/2699] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:13.996558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:14.006828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341447-1fa1-42af-b82a-0a65f0dfab5b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:14.011389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:14.012252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.014408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.014408 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] PATH_INFO: `/v3/projects/ebd29be426a64e03aa4f248d83952080/users/1ce8a032b29e4576ac259e37b695467e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.028231 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd29be426a64e03aa4f248d83952080, user_id=1ce8a032b29e4576ac259e37b695467e, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:14.031031 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:14.108584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1347/2700] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:13 2022] PUT /identity/v3/projects/ebd29be426a64e03aa4f248d83952080/users/1ce8a032b29e4576ac259e37b695467e/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:14.119794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.120181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.120468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.146347 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] MFA Rules not processed for user `1ce8a032b29e4576ac259e37b695467e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:14.148035 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:14.211263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1354/2701] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:14.222342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:14.224003 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:14.246400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:14.280666 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a49c7-b78a-4d4e-bd7c-62de8089dca6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:14.284350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:14.285266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.285727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.286356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.295784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:14.297245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c391330-fcd8-433d-9edd-dabc9f07e8bf service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:14.306486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1348/2702] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:14.331043 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:14.351250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:14.394763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416b71d-ff3c-4534-8bd6-cdadb3669272 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:14.400163 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:14.401910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.401910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.401910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] PATH_INFO: `/v3/users/1ce8a032b29e4576ac259e37b695467e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.402892 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce8a032b29e4576ac259e37b695467e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:14.404804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:14.581804 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce8a032b29e4576ac259e37b695467e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:14.591858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce8a032b29e4576ac259e37b695467e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:14.606137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce8a032b29e4576ac259e37b695467e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:14.615339 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce8a032b29e4576ac259e37b695467e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:14.709117 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce8a032b29e4576ac259e37b695467e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:14.722690 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1355/2703] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:14 2022] DELETE /identity/v3/users/1ce8a032b29e4576ac259e37b695467e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:14.728387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.728880 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.728982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.749732 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:14.750952 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:14.824100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1349/2704] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:14.834845 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:14.836371 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:14.894542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398311b2-321c-4fc6-a58f-c91fb0a2a81b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:14.899138 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:14.899787 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:14.900064 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:14.900337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:14.910092 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:14.911524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9787ad-bab5-4a0f-a061-8fdb90748503 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:14.922693 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1356/2705] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:15.350380 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:15.352738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:15.369575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e998748-bd22-48c4-ae06-afe77f2f4695 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:15.374269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:15.375158 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.375435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.375717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] PATH_INFO: `/v3/projects/ebd29be426a64e03aa4f248d83952080` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.376881 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd29be426a64e03aa4f248d83952080)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:15.378794 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:15.486101 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd29be426a64e03aa4f248d83952080', 'request_id': 'req-bf981b52-5a9a-41b9-a2d4-586258a8ff06'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:15.492905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd29be426a64e03aa4f248d83952080', 'request_id': 'req-bf981b52-5a9a-41b9-a2d4-586258a8ff06'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:15.548324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd29be426a64e03aa4f248d83952080', 'request_id': 'req-bf981b52-5a9a-41b9-a2d4-586258a8ff06'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:15.560266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] The token cache is being invalidate because project ebd29be426a64e03aa4f248d83952080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:15.560555 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:15.561208 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bf981b52-5a9a-41b9-a2d4-586258a8ff06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:15.562754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1350/2706] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:15 2022] DELETE /identity/v3/projects/ebd29be426a64e03aa4f248d83952080 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:15.567745 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.568030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.568303 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.586624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:15.587937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:15.621505 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:15.648962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1357/2707] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:15.656662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.656950 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.657263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.675379 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:15.676587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:15.685440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1351/2708] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:15.689558 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:15.699944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1b0ee-2ae7-48bc-bc39-1f6bb3fc15a2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:15.704584 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:15.705264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.705552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.705829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.706549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:00:15.706852 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:15.708437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:15.720601 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1358/2709] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:00:15.728839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:15.738352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb47d-8260-4d35-b274-92e3b9406771 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:15.743905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:15.744577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.744869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.745187 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.745882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:15.747771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:15.835224 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1352/2710] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:15 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:15.839449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:15.849395 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d702e2-40a0-4fb0-8ef0-8b85745b87ae admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:15.906360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:15.906953 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:15.907234 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:15.907507 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:15.908203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:15.910141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:16.004184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1359/2711] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:15 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:16.008705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:16.019099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bb51-4095-4a31-b135-9b1e3a26bce5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:16.024453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:16.025133 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:16.025444 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:16.025716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:16.026433 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:16.028009 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:16.038411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1353/2712] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:16.042670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:16.052518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3246d-65ab-4e48-8f99-08fa21bb955c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:16.057906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:16.059536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:16.059536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:16.059536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:16.059864 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:16.061486 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550b08d-ef77-47a2-a45a-7ac132afa036 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:16.072901 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1360/2713] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 13:00:16.076925 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:16.087725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcd9b6-1036-4028-8544-cfe3fabffcc9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:16.092403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:16.093304 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:16.093581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:16.093855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] PATH_INFO: `/v3/projects/b488641c1fce4a16a68d7086209de4e4/users/17584c17286742e5be942000d0e847c7/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:16.096106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:16.108828 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b488641c1fce4a16a68d7086209de4e4, user_id=17584c17286742e5be942000d0e847c7, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:16.111586 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:16.169188 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1354/2714] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:16 2022] PUT /identity/v3/projects/b488641c1fce4a16a68d7086209de4e4/users/17584c17286742e5be942000d0e847c7/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:16.180188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:16.180474 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:16.180749 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:16.208530 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] MFA Rules not processed for user `17584c17286742e5be942000d0e847c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:16.211171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:16.281537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1361/2715] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:16.292604 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:16.294420 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:16.354036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b870e-8489-442e-a0f2-3b82d18b25f8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:16.357725 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:16.358617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:16.358905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:16.359268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:16.368454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:16.369983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:16.379306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1355/2716] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:25.505312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:25.506705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:25.547546 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:25.581161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb144b-1ff1-45c3-885b-83482849ac84 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:25.586528 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:25.587237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:25.587519 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:25.587813 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:25.589600 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:25.650195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:25.652402 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:25.662131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1362/2717] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:26.374839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:28.172150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:28.189382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3facd2-ef19-4218-88d6-5cb6018b31a6 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:28.195218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:28.196493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:28.196878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:28.197147 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:28.199565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:28.212789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:28.214421 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:28.226265 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1356/2718] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:31.332149 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:31.333683 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:31.390060 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d08390-5493-42be-b6d1-f0fd234b992c service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:31.393751 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:31.394392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:31.394705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:31.394977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:31.396738 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:31.407051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:31.408490 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857d7aa-0cd5-40b1-9584-63545dfd64b7 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:31.417251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1363/2719] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:33.808065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:33.882945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b3d1-9162-46be-9d00-1e2a086f132f service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:33.888687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:33.889155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:33.889445 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:33.889723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] PATH_INFO: `/v3/users/17584c17286742e5be942000d0e847c7` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:33.890932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17584c17286742e5be942000d0e847c7)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:33.892850 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:34.071024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17584c17286742e5be942000d0e847c7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:34.083452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17584c17286742e5be942000d0e847c7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:34.084159 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17584c17286742e5be942000d0e847c7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:34.084798 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17584c17286742e5be942000d0e847c7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:34.092409 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17584c17286742e5be942000d0e847c7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:34.175347 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1357/2720] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:33 2022] DELETE /identity/v3/users/17584c17286742e5be942000d0e847c7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:34.179364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:34.179893 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:34.180188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:34.210332 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:34.211735 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:34.280807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1364/2721] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:34.287549 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:34.289136 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:34.353780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a3d11-b037-4d24-8581-96ddfb831ca4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:34.357661 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:34.361404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:34.361404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:34.361404 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:34.370239 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:34.371882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c27ab-5d4f-41d7-bcfe-3c449bff70b4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:34.389320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1358/2722] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:34.920225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:34.921714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:34.934136 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dda78-3b5a-4f51-aa7f-decdfa1ecf39 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:34.938958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:34.939577 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:34.939877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:34.940133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] PATH_INFO: `/v3/projects/b488641c1fce4a16a68d7086209de4e4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:34.941349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b488641c1fce4a16a68d7086209de4e4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:34.943255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:34.954879 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:35.069281 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b488641c1fce4a16a68d7086209de4e4', 'request_id': 'req-70511c41-4b0e-460b-a405-1b43cbbc6292'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:35.075955 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b488641c1fce4a16a68d7086209de4e4', 'request_id': 'req-70511c41-4b0e-460b-a405-1b43cbbc6292'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:35.142622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b488641c1fce4a16a68d7086209de4e4', 'request_id': 'req-70511c41-4b0e-460b-a405-1b43cbbc6292'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:35.154335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] The token cache is being invalidate because project b488641c1fce4a16a68d7086209de4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:35.154653 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:35.155279 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-70511c41-4b0e-460b-a405-1b43cbbc6292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:35.156823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1365/2723] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:34 2022] DELETE /identity/v3/projects/b488641c1fce4a16a68d7086209de4e4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:35.163681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.164201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.164311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.183945 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:35.185218 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:35.189702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:35.246103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1359/2724] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:35.307350 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.307942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.307942 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.326966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:35.328165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:35.336251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1366/2725] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:35.341273 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.352663 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7960c1-3cbf-4176-944b-a47e7b38e08d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.357371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.358032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.358377 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.358595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.359427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:00:35.359814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.361639 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.377331 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1360/2726] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:00:35.378566 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.388557 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeab1e4-a58f-49ad-a0d1-09405dfe4aa1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.393439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.394096 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.394381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.394658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.395349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.397290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.528501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1367/2727] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:35 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:35.532459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.544922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8406-fe1b-4b2c-9995-83236f771c80 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.627109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.628232 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.629362 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.629902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.630912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.634412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.735679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1361/2728] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:35 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 13:00:35.739743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.750493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db3adb-0bbb-4239-8cdb-d0dced809f15 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.755184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.755840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.756196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.756477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.757236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.758823 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.771053 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1368/2729] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:35.775743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.792049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f8a6-b4b7-4713-9065-15547ed026fa admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.796836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.797994 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.798301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.798602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.799324 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.800915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40297aa-f3fe-43b3-9340-fe544ba12a39 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.813316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1362/2730] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 13:00:35.817130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:35.828216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19e66-354f-4ff6-bb28-46d87b1a193b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:35.832924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:35.833829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.834110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.834386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] PATH_INFO: `/v3/projects/ada92e33c07f4be7b2d752550c381dfb/users/c89a6bacbd004d99837d8d092352f18f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.848913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada92e33c07f4be7b2d752550c381dfb, user_id=c89a6bacbd004d99837d8d092352f18f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:35.851651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:35.931809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1369/2731] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:35 2022] PUT /identity/v3/projects/ada92e33c07f4be7b2d752550c381dfb/users/c89a6bacbd004d99837d8d092352f18f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:35.937429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:35.937829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:35.937983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:35.956482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:35.957702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:36.021617 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1363/2732] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:36.031918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:36.033665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:36.091534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225f007-bde2-4017-aaf6-e3e0f37f996b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:36.095261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:36.095909 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:36.096198 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:36.096476 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:36.105789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:36.107256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44e131-b7f9-4ba5-8562-1c74ba5b56eb service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:36.116650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1370/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:37.099896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:37.109826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592234c-b812-441e-9c9a-0ef46c4a389a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:37.113727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:37.114367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:37.114649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:37.114927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:37.116735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:37.129720 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:37.131123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:37.141843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1364/2734] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:37.332995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76236-8bb2-422d-bbe5-e5d46215252a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:37.333527 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76236-8bb2-422d-bbe5-e5d46215252a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:37.333682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76236-8bb2-422d-bbe5-e5d46215252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:37.375042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c0d76236-8bb2-422d-bbe5-e5d46215252a None None] MFA Rules not processed for user `c89a6bacbd004d99837d8d092352f18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:37.376340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c0d76236-8bb2-422d-bbe5-e5d46215252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:37.405378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:37.449296 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1371/2735] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:37.465381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:37.465381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:37.467088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:37.538900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34458e-5b09-44d3-985d-d1b786933b13 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:37.543039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:37.543683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:37.543966 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:37.544853 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:37.553139 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:37.554927 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:37.564241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1365/2736] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:39.379421 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64812b8-fede-47e6-934c-15864443e12c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:39.380294 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64812b8-fede-47e6-934c-15864443e12c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:39.380588 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64812b8-fede-47e6-934c-15864443e12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:39.398489 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f64812b8-fede-47e6-934c-15864443e12c None None] MFA Rules not processed for user `c89a6bacbd004d99837d8d092352f18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:39.399695 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f64812b8-fede-47e6-934c-15864443e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:39.407529 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1372/2737] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:39.667834 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:39.669702 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:39.686071 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2e359-0df2-48dd-be5d-4ccace00d6a6 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:39.694885 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:39.703809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:39.704412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:39.704848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:39.707075 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:39.720937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:39.722564 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:39.732848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1366/2738] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:40.164988 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64812b8-fede-47e6-934c-15864443e12c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:40.166427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f64812b8-fede-47e6-934c-15864443e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:40.278218 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64812b8-fede-47e6-934c-15864443e12c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:40.283235 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:40.283981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:40.284911 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:40.290865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:40.290865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:40.306792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:40.309559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:40.326019 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1373/2739] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:44.739115 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:44.742482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:44.758864 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562db62-ec2e-49dc-8601-8bab86d4ea32 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:44.763451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:44.764402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:44.764687 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:44.764965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] PATH_INFO: `/v3/users/8431eaa3717a4f798db80c9c87832273` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:44.766195 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8431eaa3717a4f798db80c9c87832273)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:44.768118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:44.961644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8431eaa3717a4f798db80c9c87832273'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:44.974105 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8431eaa3717a4f798db80c9c87832273'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:44.974771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8431eaa3717a4f798db80c9c87832273'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:44.975402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8431eaa3717a4f798db80c9c87832273'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:44.983342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8431eaa3717a4f798db80c9c87832273'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.066766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1367/2740] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:44 2022] DELETE /identity/v3/users/8431eaa3717a4f798db80c9c87832273 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:45.321247 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:45.342670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb9ba1-f00d-4f7f-ac8b-e70f3fee5dea service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:45.417440 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:45.418422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:45.418709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:45.422029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] PATH_INFO: `/v3/projects/73deeaa0bcd7498eae06c39dfc0e2e3b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:45.422029 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73deeaa0bcd7498eae06c39dfc0e2e3b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:45.422226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:45.516980 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73deeaa0bcd7498eae06c39dfc0e2e3b', 'request_id': 'req-ec461506-f14b-47d3-a4fa-c0cc8e155444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.520017 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:45.528442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73deeaa0bcd7498eae06c39dfc0e2e3b', 'request_id': 'req-ec461506-f14b-47d3-a4fa-c0cc8e155444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.612180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73deeaa0bcd7498eae06c39dfc0e2e3b', 'request_id': 'req-ec461506-f14b-47d3-a4fa-c0cc8e155444'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.629966 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] The token cache is being invalidate because project 73deeaa0bcd7498eae06c39dfc0e2e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:45.630261 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.630916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.632757 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1374/2741] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:45 2022] DELETE /identity/v3/projects/73deeaa0bcd7498eae06c39dfc0e2e3b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:45.637310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:45.639201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:45.654797 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc46606-49fb-48be-b29d-5c7079fd999b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:45.744283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:45.746587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:45.747168 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:45.747915 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] PATH_INFO: `/v3/users/8ef5ec397a8a44fda3644476b2f66586` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:45.749536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef5ec397a8a44fda3644476b2f66586)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:45.751796 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:45.759318 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:45.937803 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef5ec397a8a44fda3644476b2f66586'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.951171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef5ec397a8a44fda3644476b2f66586'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.951998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef5ec397a8a44fda3644476b2f66586'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.952699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef5ec397a8a44fda3644476b2f66586'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:45.960193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef5ec397a8a44fda3644476b2f66586'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.028733 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1368/2742] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:45 2022] DELETE /identity/v3/users/8ef5ec397a8a44fda3644476b2f66586 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:46.282336 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:46.284427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:46.306128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec461506-f14b-47d3-a4fa-c0cc8e155444 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:46.368711 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:46.369533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:46.369846 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:46.370211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] PATH_INFO: `/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:46.371425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b483ffd8ef6e41708af269ea8ad3d6bf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:46.373351 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:46.485797 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b483ffd8ef6e41708af269ea8ad3d6bf', 'request_id': 'req-707a8c28-383c-41b3-acab-2e0c5ca9ea83'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.492709 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b483ffd8ef6e41708af269ea8ad3d6bf', 'request_id': 'req-707a8c28-383c-41b3-acab-2e0c5ca9ea83'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.576984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b483ffd8ef6e41708af269ea8ad3d6bf', 'request_id': 'req-707a8c28-383c-41b3-acab-2e0c5ca9ea83'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.591120 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] The token cache is being invalidate because project b483ffd8ef6e41708af269ea8ad3d6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:46.591424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.592055 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-707a8c28-383c-41b3-acab-2e0c5ca9ea83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:46.593641 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1375/2743] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:46 2022] DELETE /identity/v3/projects/b483ffd8ef6e41708af269ea8ad3d6bf => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:50.024902 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:50.026911 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:50.039490 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c127f-0d4d-4aa7-a86c-508b744185b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:50.095418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:50.096283 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:50.096562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:50.096833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] PATH_INFO: `/v3/users/e87b609824ac4c6685e218e2d5a009b5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:50.098021 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87b609824ac4c6685e218e2d5a009b5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:50.100316 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:50.257602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87b609824ac4c6685e218e2d5a009b5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:50.270167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87b609824ac4c6685e218e2d5a009b5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:50.270817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87b609824ac4c6685e218e2d5a009b5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:50.271406 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87b609824ac4c6685e218e2d5a009b5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:50.279610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87b609824ac4c6685e218e2d5a009b5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:50.365419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1369/2744] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:50 2022] DELETE /identity/v3/users/e87b609824ac4c6685e218e2d5a009b5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:50.370867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:50.371364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:50.371478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:50.390916 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:50.392221 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:50.454256 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1376/2745] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:50.464452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:50.466019 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:50.471700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:50.524837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca0e3-d51e-40f3-8ca2-8444d57d3395 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:50.529615 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:50.534844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:50.534844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:50.534844 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:50.539688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:50.541174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:50.551260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1370/2746] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:51.046456 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:51.047984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:51.060725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc285-50c4-4785-9691-92ae145896d3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:51.066099 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:51.067008 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:51.067295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:51.067597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] PATH_INFO: `/v3/projects/f74dc4d8393e482b8b180b33ba7de582` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:51.068795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74dc4d8393e482b8b180b33ba7de582)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:51.070822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:51.078503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:51.178109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74dc4d8393e482b8b180b33ba7de582', 'request_id': 'req-6f402f79-b145-4b5f-874a-c9ed199c5ffc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.185730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74dc4d8393e482b8b180b33ba7de582', 'request_id': 'req-6f402f79-b145-4b5f-874a-c9ed199c5ffc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.268045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74dc4d8393e482b8b180b33ba7de582', 'request_id': 'req-6f402f79-b145-4b5f-874a-c9ed199c5ffc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.278878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] The token cache is being invalidate because project f74dc4d8393e482b8b180b33ba7de582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:51.279382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.280484 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.282215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1377/2747] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:51 2022] DELETE /identity/v3/projects/f74dc4d8393e482b8b180b33ba7de582 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:51.286274 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:51.296610 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6cc48-b3c5-4359-89d0-ece424e74b06 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:51.353266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:51.354069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:51.354351 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:51.354627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] PATH_INFO: `/v3/users/8157407d28464e419358ee8d2e59c216` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:51.356473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8157407d28464e419358ee8d2e59c216)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:51.358441 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:51.590556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8157407d28464e419358ee8d2e59c216'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.603710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8157407d28464e419358ee8d2e59c216'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.604342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8157407d28464e419358ee8d2e59c216'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.605728 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8157407d28464e419358ee8d2e59c216'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.612887 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-44dfef33-a363-4658-a208-05c7a6234db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8157407d28464e419358ee8d2e59c216'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:51.735268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1371/2748] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:51 2022] DELETE /identity/v3/users/8157407d28464e419358ee8d2e59c216 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:52.148119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.149730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:52.170867 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f402f79-b145-4b5f-874a-c9ed199c5ffc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.231743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.232597 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.232882 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.233195 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] PATH_INFO: `/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.234372 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd4eaa308a54662a3d5365b4271b1ef)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.236326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.334630 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd4eaa308a54662a3d5365b4271b1ef', 'request_id': 'req-24f76f51-4cc4-435c-a472-2b1fe945e34c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:52.343480 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd4eaa308a54662a3d5365b4271b1ef', 'request_id': 'req-24f76f51-4cc4-435c-a472-2b1fe945e34c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:52.434210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd4eaa308a54662a3d5365b4271b1ef', 'request_id': 'req-24f76f51-4cc4-435c-a472-2b1fe945e34c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:52.445178 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] The token cache is being invalidate because project dcd4eaa308a54662a3d5365b4271b1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:52.445464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:52.446086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24f76f51-4cc4-435c-a472-2b1fe945e34c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:52.447669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1378/2749] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:52 2022] DELETE /identity/v3/projects/dcd4eaa308a54662a3d5365b4271b1ef => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:52.453375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.454527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.454527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.473166 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:52.474385 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:52.494705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:52.537579 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1372/2750] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:52.546914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.547208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.547477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.567865 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:52.569364 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:52.577914 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1379/2751] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:52.582244 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.592103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b0035-68bc-44fd-b9f5-cec421fe749a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.596949 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.597713 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.598016 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.599237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.600057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:00:52.600394 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.602030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.614826 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1373/2752] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:00:52.619982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.630677 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a513e-6d84-4f36-aa9f-90032c96a2e1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.635378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.638222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.638222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.638222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.638222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.639337 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.653686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:52.713143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1380/2753] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:52 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:52.714267 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.725086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b32f02-a333-4641-9f19-72d72b8d17e6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.785180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.785899 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.786183 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.786454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.787144 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.789073 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.863176 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1374/2754] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:52 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:52.867532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.878923 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c833ce-48ec-4518-a640-429680f4611b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.883459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.884102 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.884385 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.884658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.885409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.887464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.897629 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1381/2755] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:52.902971 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.912778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece67c-2cbb-4347-a5b8-1fcd64352440 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.917907 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.919088 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.919371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.919649 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.920364 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.921977 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd9134-a140-49c8-bfcc-933d89d6a726 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:52.931921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1375/2756] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:52.936642 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:52.946494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fba34c-eebb-4ae7-bb03-09759f09be9e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:52.951128 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:52.951995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:52.952277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:52.952549 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] PATH_INFO: `/v3/projects/6b57665988d94446af36afe96707d65c/users/28ff05d1a77845ea9e3cb21343d53cf3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:52.967301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b57665988d94446af36afe96707d65c, user_id=28ff05d1a77845ea9e3cb21343d53cf3, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:52.969935 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.039789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1382/2757] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:52 2022] PUT /identity/v3/projects/6b57665988d94446af36afe96707d65c/users/28ff05d1a77845ea9e3cb21343d53cf3/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:53.050923 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.051214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.051489 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.081006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] MFA Rules not processed for user `28ff05d1a77845ea9e3cb21343d53cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:53.082245 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:53.148011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1376/2758] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 13:00:53.153634 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.218575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30e03-61cf-44d3-8d20-a54c8162c4d9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.223497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.224151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.224430 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.224705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.225429 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.227315 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.344259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1383/2759] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:53 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:53.348233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.359371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ab535-21f6-4fbc-ace5-40dc3d6d8e93 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.375818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:53.415234 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.415894 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.416174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.416448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.417365 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.419123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.519006 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1377/2760] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:53 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:53.522930 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.534686 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34ef97-104b-4595-adac-7a0fe2ecf00e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.539323 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.539977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.540382 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.540682 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.541447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.543762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.677481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1384/2761] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:53.682065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.692270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6a07d-d791-463e-b59f-beb39a95b111 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.696800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.697705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.697987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.698261 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] PATH_INFO: `/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df/users/947d0e11f8494533be3da0c2c65e9640/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.712869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c58ec92fc1e48e7af6fb34a3e3dd0df, user_id=947d0e11f8494533be3da0c2c65e9640, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.716086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.777138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1378/2762] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:53 2022] PUT /identity/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df/users/947d0e11f8494533be3da0c2c65e9640/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:53.780945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.790904 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0362bb-428e-4f56-8551-86102fc09694 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.846637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.847298 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.847573 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.847839 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.848650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.850146 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bb71e-13aa-4cee-a40a-88a3d0418e4e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.865341 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1385/2763] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:53.870566 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:53.878862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58dadf-0284-443a-8479-1b3c8c720212 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:53.888749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:53.889830 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.890112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.890705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] PATH_INFO: `/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df/users/947d0e11f8494533be3da0c2c65e9640/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.892392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c58ec92fc1e48e7af6fb34a3e3dd0df, user_id=947d0e11f8494533be3da0c2c65e9640, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:53.895096 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:53.907471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1379/2764] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:53 2022] PUT /identity/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df/users/947d0e11f8494533be3da0c2c65e9640/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:53.918816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:53.921919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:53.921919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:53.949357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] MFA Rules not processed for user `947d0e11f8494533be3da0c2c65e9640`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:53.949357 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:53.968627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:54.022394 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1386/2765] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:54.034298 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:54.038209 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:54.097329 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39967df-b4ad-497a-897a-3519805bccb9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:54.101268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:54.103543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:54.103543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:54.103543 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:54.107262 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:54.108861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:54.117878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:54.119558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:54.130259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1380/2766] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:54.158651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:54.187076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c4e82-7268-4df0-9754-04a93412a2fe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:54.193059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:54.193726 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:54.194009 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:54.194280 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:54.196186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:54.209354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:54.211977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a2954-c607-455e-a953-585bf2f3bc93 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:54.224424 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1387/2767] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:54.234743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727cc38-d582-4605-be6a-c52b88a9ac2b service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:54.240569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:54.241729 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:54.242011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:54.242276 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] PATH_INFO: `/v3/users/28ff05d1a77845ea9e3cb21343d53cf3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:54.243508 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ff05d1a77845ea9e3cb21343d53cf3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:54.246163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:54.264722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:54.430371 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ff05d1a77845ea9e3cb21343d53cf3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:54.443763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ff05d1a77845ea9e3cb21343d53cf3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:54.444480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ff05d1a77845ea9e3cb21343d53cf3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:54.445112 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ff05d1a77845ea9e3cb21343d53cf3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:54.451993 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ff05d1a77845ea9e3cb21343d53cf3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:54.535792 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1381/2768] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:54 2022] DELETE /identity/v3/users/28ff05d1a77845ea9e3cb21343d53cf3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:54.539951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:54.540263 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:54.540542 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:54.559990 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:54.561249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:54.624199 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1388/2769] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:54.633452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:54.634921 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:54.693199 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fef87-16af-4e41-b23d-1f1cb4a12bf6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:54.696712 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:54.697386 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:54.697669 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:54.697944 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:54.707533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:54.708953 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:54.718575 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1382/2770] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:00:55.115670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:55.117406 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:55.131871 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbdb25-e4a9-44ca-9944-bea1fdb52671 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:55.137611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:55.138969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:55.139260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:55.139537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] PATH_INFO: `/v3/projects/6b57665988d94446af36afe96707d65c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:55.140920 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b57665988d94446af36afe96707d65c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:55.143962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:55.260151 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:55.268444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b57665988d94446af36afe96707d65c', 'request_id': 'req-94daddba-983f-4ed8-9c5d-dd6172aff783'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.274520 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b57665988d94446af36afe96707d65c', 'request_id': 'req-94daddba-983f-4ed8-9c5d-dd6172aff783'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.353249 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b57665988d94446af36afe96707d65c', 'request_id': 'req-94daddba-983f-4ed8-9c5d-dd6172aff783'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.364352 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] The token cache is being invalidate because project 6b57665988d94446af36afe96707d65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:55.365229 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.365889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.367419 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1389/2771] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:55 2022] DELETE /identity/v3/projects/6b57665988d94446af36afe96707d65c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:55.372116 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:55.381843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6b87-aa99-4e30-be90-dbb23753229c service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:55.437391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:55.438252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:55.438534 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:55.438808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] PATH_INFO: `/v3/users/947d0e11f8494533be3da0c2c65e9640` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:55.439980 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=947d0e11f8494533be3da0c2c65e9640)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:55.441917 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:55.562525 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:55.568429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947d0e11f8494533be3da0c2c65e9640'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.580099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947d0e11f8494533be3da0c2c65e9640'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.588620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947d0e11f8494533be3da0c2c65e9640'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.588620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947d0e11f8494533be3da0c2c65e9640'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.588620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-67dce601-4d30-45c5-8f40-d657e5f32ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947d0e11f8494533be3da0c2c65e9640'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:55.668310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1383/2772] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:00:55 2022] DELETE /identity/v3/users/947d0e11f8494533be3da0c2c65e9640 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:56.092945 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.094485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:56.108455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daddba-983f-4ed8-9c5d-dd6172aff783 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.168366 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.169439 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.169802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.170180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] PATH_INFO: `/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.171509 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c58ec92fc1e48e7af6fb34a3e3dd0df)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.173552 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.303142 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c58ec92fc1e48e7af6fb34a3e3dd0df', 'request_id': 'req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:56.309732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c58ec92fc1e48e7af6fb34a3e3dd0df', 'request_id': 'req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:56.388673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c58ec92fc1e48e7af6fb34a3e3dd0df', 'request_id': 'req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:56.401119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] The token cache is being invalidate because project 4c58ec92fc1e48e7af6fb34a3e3dd0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:00:56.401404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:56.402789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2eb4f35-b7d2-4add-8249-63d8a3e4a1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:00:56.404282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1390/2773] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:00:56 2022] DELETE /identity/v3/projects/4c58ec92fc1e48e7af6fb34a3e3dd0df => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:56.410034 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.410327 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.410622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.428391 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:56.429884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:56.490241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1384/2774] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:56.499386 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.499743 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.499982 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.519330 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:56.520539 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:56.528761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1391/2775] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:56.535039 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.543589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915381d-4512-4b3a-b4e8-f894c1be7677 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.548557 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.549727 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.550104 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.550427 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.551241 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:00:56.551546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.553301 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.566674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1385/2776] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:00:56.571295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.580604 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f379c8-3ff7-45d2-b344-1d5383082ab5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.585290 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.585933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.586245 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.586543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.587284 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.589701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.686516 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1392/2777] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:56 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:56.690714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.701015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce78f6-b21a-4d25-b414-3c80ea265893 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.756486 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.757177 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.757463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.757739 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.758416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.760309 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.821854 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1386/2778] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:56 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:56.826345 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.836282 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792eecf-7823-4e66-907f-903d5d0ea6bd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.841664 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.842246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.842524 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.842792 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.843497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.845076 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.856023 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1393/2779] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:56.859824 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.869785 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba98c91-3f72-442d-9a21-9eeb80ac2c5a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.874599 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.879469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.879469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.879469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.879469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.879469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea41cd-2c7f-4769-a4da-09bf0a13746e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:56.887935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1387/2780] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:56.891878 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:56.901984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3516d-c5e5-4342-9c37-840aed8447da admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:56.906733 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:56.907582 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:56.907858 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:56.908127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] PATH_INFO: `/v3/projects/9db9365756bb46abadd84fc7c852676b/users/86f55dfdac9d474c9a036b9377f326db/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:56.920369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:56.923448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db9365756bb46abadd84fc7c852676b, user_id=86f55dfdac9d474c9a036b9377f326db, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:56.926171 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.033606 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1394/2781] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:56 2022] PUT /identity/v3/projects/9db9365756bb46abadd84fc7c852676b/users/86f55dfdac9d474c9a036b9377f326db/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:57.044674 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.044963 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.045275 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.073363 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] MFA Rules not processed for user `86f55dfdac9d474c9a036b9377f326db`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:57.074886 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:57.084024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:57.139760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1388/2782] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:57.144405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.212616 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac821d-e0ab-4a1b-be82-7d051020c817 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.217518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.218176 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.218457 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.218730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.219412 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.221360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.326329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1395/2783] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:00:57 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:57.330550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.341790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aef30-241b-4fff-a68a-121beb09d480 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.400487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.400487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.400809 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.401023 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.402259 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.404233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.487684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1389/2784] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:00:57 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:00:57.489205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.499663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb694-35af-4929-b921-626494d9dc28 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.504475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.505180 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.505464 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.510272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.510272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.510272 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.519746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1396/2785] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:57.526258 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.538959 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359df03-65df-42e6-a363-29f7f2190a0b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.545438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.546938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.547325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.547667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] PATH_INFO: `/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80/users/923772445d03437bb18c190807a85239/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.565811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f2b4e1ad664a5fa55698309dfcfd80, user_id=923772445d03437bb18c190807a85239, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.569206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.660816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1390/2786] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:57 2022] PUT /identity/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80/users/923772445d03437bb18c190807a85239/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:00:57.662720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.674425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2404-5904-4f29-962c-11bc8a216036 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.739712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.740381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.740658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.740931 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.742414 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.744000 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878a6c5-efdd-4274-a5cf-966dd2640451 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.754131 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1397/2787] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:00:57.758412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.772119 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af445149-6685-40c6-8651-7ea56285d089 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.778473 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.780155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.780155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.780155 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] PATH_INFO: `/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80/users/923772445d03437bb18c190807a85239/roles/a495d38f67e447309647ac782785bd87` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.784722 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f2b4e1ad664a5fa55698309dfcfd80, user_id=923772445d03437bb18c190807a85239, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:57.785477 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:57.800352 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1391/2788] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:00:57 2022] PUT /identity/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80/users/923772445d03437bb18c190807a85239/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:00:57.813277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.813277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.813277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:57.841434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] MFA Rules not processed for user `923772445d03437bb18c190807a85239`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:00:57.842959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:57.891226 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:57.912828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1398/2789] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:00:57.923882 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:00:57.925940 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:00:57.985646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ac70d-ad3e-4a18-89e5-6d0d05a88e13 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:00:57.989483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:00:57.990125 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:00:57.990413 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:00:57.990690 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:00:58.016756 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:00:58.053754 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:00:58.055470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:00:58.065310 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1392/2790] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:00:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:03.194119 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:03.204037 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff28d-d7d0-4928-b1bf-60e4de0f4736 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:03.207595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:03.208224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:03.208506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:03.208785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:03.217550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:03.218958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c22de-6cbd-42e6-a958-c0111ba75962 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:03.228188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1399/2791] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:08.757761 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=19 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:09.211587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:09.280345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d839-01ee-4227-a5dd-f73562d4ab86 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:09.285052 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:09.285922 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:09.286202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:09.286478 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] PATH_INFO: `/v3/users/86f55dfdac9d474c9a036b9377f326db` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:09.287681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f55dfdac9d474c9a036b9377f326db)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:09.289680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:09.481815 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f55dfdac9d474c9a036b9377f326db'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:09.494749 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f55dfdac9d474c9a036b9377f326db'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:09.495438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f55dfdac9d474c9a036b9377f326db'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:09.496046 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f55dfdac9d474c9a036b9377f326db'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:09.503001 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f55dfdac9d474c9a036b9377f326db'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:09.576531 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1393/2792] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:01:09 2022] DELETE /identity/v3/users/86f55dfdac9d474c9a036b9377f326db => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:09.581257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:09.581775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:09.581900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:09.599637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:09.603685 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:09.664497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1400/2793] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:09.674973 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:09.676369 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:09.735126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b6405-8506-47c6-b8ee-e9d03a700393 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:09.738735 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:09.739381 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:09.739658 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:09.739935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:09.750051 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:09.751457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:09.761282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1394/2794] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:10.194161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:10.195624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:10.207591 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f42aa-2b1d-41e0-8336-acdcaab8a74c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:10.212437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:10.213399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:10.213763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:10.214093 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] PATH_INFO: `/v3/projects/9db9365756bb46abadd84fc7c852676b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:10.215318 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db9365756bb46abadd84fc7c852676b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:10.217300 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:10.318157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db9365756bb46abadd84fc7c852676b', 'request_id': 'req-2ac6a605-e06a-4633-972d-36e3f4f21080'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.324040 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db9365756bb46abadd84fc7c852676b', 'request_id': 'req-2ac6a605-e06a-4633-972d-36e3f4f21080'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.397832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db9365756bb46abadd84fc7c852676b', 'request_id': 'req-2ac6a605-e06a-4633-972d-36e3f4f21080'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.408977 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] The token cache is being invalidate because project 9db9365756bb46abadd84fc7c852676b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:01:10.409333 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.409981 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.411533 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1401/2795] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:01:10 2022] DELETE /identity/v3/projects/9db9365756bb46abadd84fc7c852676b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:10.415505 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:10.425556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb3fd0-cb48-40e5-ba4b-c47748c7e15f service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:10.444375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:10.483089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:10.484544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:10.484859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:10.485167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] PATH_INFO: `/v3/users/923772445d03437bb18c190807a85239` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:10.486350 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923772445d03437bb18c190807a85239)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:10.488285 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:10.663908 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923772445d03437bb18c190807a85239'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.676429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923772445d03437bb18c190807a85239'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.677129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923772445d03437bb18c190807a85239'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.677845 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923772445d03437bb18c190807a85239'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.684946 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7c3390e-836b-46e9-b838-4b6475f41dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923772445d03437bb18c190807a85239'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:10.764680 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1395/2796] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:01:10 2022] DELETE /identity/v3/users/923772445d03437bb18c190807a85239 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:11.183840 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.185349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:11.199478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6a605-e06a-4633-972d-36e3f4f21080 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.251423 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:11.257387 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.258312 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.258622 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.258896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] PATH_INFO: `/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.260116 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f2b4e1ad664a5fa55698309dfcfd80)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.262056 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.359627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f2b4e1ad664a5fa55698309dfcfd80', 'request_id': 'req-6d041c56-8ded-480b-983f-ebc8996e079a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:11.365671 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f2b4e1ad664a5fa55698309dfcfd80', 'request_id': 'req-6d041c56-8ded-480b-983f-ebc8996e079a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:11.427384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f2b4e1ad664a5fa55698309dfcfd80', 'request_id': 'req-6d041c56-8ded-480b-983f-ebc8996e079a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:11.439086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] The token cache is being invalidate because project 84f2b4e1ad664a5fa55698309dfcfd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:01:11.441127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:11.441127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d041c56-8ded-480b-983f-ebc8996e079a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:11.441825 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1402/2797] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:01:11 2022] DELETE /identity/v3/projects/84f2b4e1ad664a5fa55698309dfcfd80 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:11.447860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.448100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.448397 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.467029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:11.468387 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:11.527855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1396/2798] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:11.536746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.537058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.537369 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.556042 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:11.557283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:11.565536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1403/2799] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:11.569716 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.579581 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec79f-c465-4c39-b3f8-34dfb7395ae2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.584252 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.584937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.585248 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.585528 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.586260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:01:11.586598 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.588214 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.600812 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1397/2800] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:01:11.605259 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.615086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a13ce-ec39-4c73-8b1d-dfeb9ed54973 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.619732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.620373 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.620728 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.621006 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.621720 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.623658 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.697213 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1404/2801] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:01:11 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:01:11.701418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.711322 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9d322-ade3-4b94-9066-900940c94d7e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.740683 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:11.770839 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.771507 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.771789 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.772109 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.772863 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.774837 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.854556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1398/2802] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:01:11 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:01:11.858673 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.869236 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96255-624b-48a3-94f6-631daaf2334d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.873960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.874614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.874899 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.875205 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.876135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.877764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.888396 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1405/2803] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:01:11.893286 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.903345 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03a37-13ed-48ce-998b-f0df5dbb70aa admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.908984 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.909679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.909968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.910269 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.911150 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.912765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1b0e9-560a-4f71-83ae-0d7bbf74f6e9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:11.923342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1399/2804] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:01:11.927378 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:11.937390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5e3f3-4bae-4200-8e9c-26179ed2a9ef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:11.942210 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:11.943058 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:11.943349 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:11.943624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] PATH_INFO: `/v3/projects/3ea35c288411485e84f42dab6d2d407f/users/148142945fd649d6b2525df8dd2be1c0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:11.958984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea35c288411485e84f42dab6d2d407f, user_id=148142945fd649d6b2525df8dd2be1c0, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:11.961578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:12.049442 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1406/2805] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:01:11 2022] PUT /identity/v3/projects/3ea35c288411485e84f42dab6d2d407f/users/148142945fd649d6b2525df8dd2be1c0/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:12.060805 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:12.061121 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:12.061400 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:12.088036 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] MFA Rules not processed for user `148142945fd649d6b2525df8dd2be1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:12.089264 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:12.151333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1400/2806] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:12.163045 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:12.164713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:12.222707 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06dc2e-d13f-4f3a-9d86-53bfbd24bae6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:12.226209 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:12.226848 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:12.227132 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:12.227408 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:12.236231 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:12.237713 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8dd15-3b8e-4363-b9c5-c044008b524a service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:12.246344 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1407/2807] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:17.173499 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:17.238679 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707e44-4df3-4308-a076-7b5c8a779e17 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:17.243700 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:17.244587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:17.244930 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:17.245235 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] PATH_INFO: `/v3/users/148142945fd649d6b2525df8dd2be1c0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:17.246442 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=148142945fd649d6b2525df8dd2be1c0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:17.248355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:17.389965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:17.396343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148142945fd649d6b2525df8dd2be1c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:17.408493 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148142945fd649d6b2525df8dd2be1c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:17.409297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148142945fd649d6b2525df8dd2be1c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:17.409936 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148142945fd649d6b2525df8dd2be1c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:17.416799 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148142945fd649d6b2525df8dd2be1c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:17.478459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1401/2808] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:01:17 2022] DELETE /identity/v3/users/148142945fd649d6b2525df8dd2be1c0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:01:17.483061 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:17.483534 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:17.483678 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:17.501621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:17.502887 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:17.523051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:17.561468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1408/2809] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:17.571612 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:17.573161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:17.630724 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86e09d-2608-40f5-a5b7-71cc317c4b2d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:17.634439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:17.637538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:17.637538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:17.637538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:17.644800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:17.646202 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe86f91-22a1-4eb2-9d63-590dfc3a7bb3 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:17.655956 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1402/2810] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:18.053004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.054761 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:18.068767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4701-46bc-4e5a-994d-600721c26715 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.073665 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.074770 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.075200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.075482 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] PATH_INFO: `/v3/projects/3ea35c288411485e84f42dab6d2d407f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.076670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea35c288411485e84f42dab6d2d407f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.079246 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.210073 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea35c288411485e84f42dab6d2d407f', 'request_id': 'req-4006d098-feea-493e-a1b3-facb55848a38'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:18.221986 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea35c288411485e84f42dab6d2d407f', 'request_id': 'req-4006d098-feea-493e-a1b3-facb55848a38'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:18.284984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea35c288411485e84f42dab6d2d407f', 'request_id': 'req-4006d098-feea-493e-a1b3-facb55848a38'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:18.295821 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] The token cache is being invalidate because project 3ea35c288411485e84f42dab6d2d407f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:01:18.296109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:18.296739 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4006d098-feea-493e-a1b3-facb55848a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:01:18.299187 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1409/2811] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:01:18 2022] DELETE /identity/v3/projects/3ea35c288411485e84f42dab6d2d407f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:01:18.304356 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541672-2090-47c5-a2fb-695630099825 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.304829 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541672-2090-47c5-a2fb-695630099825 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.304938 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541672-2090-47c5-a2fb-695630099825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.322694 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-45541672-2090-47c5-a2fb-695630099825 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:18.327282 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-45541672-2090-47c5-a2fb-695630099825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:18.382558 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1403/2812] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:18.391355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.391649 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.391959 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.410956 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:18.412214 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:18.420130 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1410/2813] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:18.424896 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541672-2090-47c5-a2fb-695630099825 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.435095 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541672-2090-47c5-a2fb-695630099825 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.439651 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.440297 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.440583 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.440860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.441620 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:01:18.441958 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.443542 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.455289 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1404/2814] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:01:18.459493 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.468637 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c10f39-23ad-4f9a-b9fa-a180ad50141c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.473928 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.474574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.474855 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.475126 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.475802 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.477807 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.588862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1411/2815] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:01:18 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 13:01:18.593182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.605573 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe8992-a640-451f-9243-b3a0123b8dd4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.650461 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:18.663277 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.664801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.664801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.664801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.665170 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.667057 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.764808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1405/2816] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:01:18 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:01:18.769646 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.780410 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022d0c-086d-474a-aa7c-47e388358b4d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.786068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.786714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.786993 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.787264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.787972 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.789586 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.800222 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1412/2817] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:01:18.804857 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.814816 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4a65-ff37-487b-a26c-4817977bc25e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.819932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.820580 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.820861 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.821167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.821884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.823452 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0930-545c-419f-9cbc-5d4a3be579ac admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.834033 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1406/2818] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:01:18.838217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:18.848969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b360a0-7171-48b6-8398-d3aa97efe282 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:18.853580 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:18.854445 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.854723 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.854996 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] PATH_INFO: `/v3/projects/a5927ebc582d49538443e497cc15519d/users/b68e5780b6324fc48b45b38503d2bcb6/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.869334 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5927ebc582d49538443e497cc15519d, user_id=b68e5780b6324fc48b45b38503d2bcb6, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:18.871936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:18.947608 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1413/2819] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:01:18 2022] PUT /identity/v3/projects/a5927ebc582d49538443e497cc15519d/users/b68e5780b6324fc48b45b38503d2bcb6/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:01:18.952878 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcce56-eded-467a-9f14-455bdeb3df78 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:18.953206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcce56-eded-467a-9f14-455bdeb3df78 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:18.953488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcce56-eded-467a-9f14-455bdeb3df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:18.972562 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-07dcce56-eded-467a-9f14-455bdeb3df78 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:18.974099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-07dcce56-eded-467a-9f14-455bdeb3df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:19.032236 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1407/2820] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:19.042650 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:19.044329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:19.095651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:19.101348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0294dd-e872-4e03-99a4-72d28fe5c3d0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:19.104995 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:19.105663 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:19.105944 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:19.106217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:19.115718 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:19.117157 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:19.126441 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1414/2821] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:20.423142 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64bda-b515-4f7c-922b-2348077e01b6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:20.423684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64bda-b515-4f7c-922b-2348077e01b6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:20.423684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64bda-b515-4f7c-922b-2348077e01b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:20.451760 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-95d64bda-b515-4f7c-922b-2348077e01b6 None None] MFA Rules not processed for user `b68e5780b6324fc48b45b38503d2bcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:20.453103 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-95d64bda-b515-4f7c-922b-2348077e01b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:20.521929 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1408/2822] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:20.534403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:20.535938 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:20.594485 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9107f-a1dc-4bd4-b90b-f362b2c34317 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:20.598829 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:20.600075 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:20.601391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:20.601391 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:20.609578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:20.610991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fec57-de41-4c5e-bba4-2704553076b7 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:20.620428 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1415/2823] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:01:22.351587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:22.352089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:22.352418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:22.371268 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] MFA Rules not processed for user `b68e5780b6324fc48b45b38503d2bcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:22.372460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:22.379790 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1409/2824] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:30.616459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=31 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:32.364913 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:01:51.855013 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:51.855628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:51.855628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:51.872860 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] MFA Rules not processed for user `b68e5780b6324fc48b45b38503d2bcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:01:51.874212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:51.880833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1416/2825] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:01:56.988706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:01:56.990240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:57.091163 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f674abb-61ba-414f-8aff-45835931a75d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:01:57.096836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:01:57.097523 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:01:57.097806 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:01:57.098079 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:01:57.100049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:01:57.112416 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:01:57.114354 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314705c1-a501-4935-a400-c0f967ed9a3a service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:01:57.125174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1410/2826] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:01:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:02:01.868135 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:04.375172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:04.376674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:04.446405 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e391272-ae2d-4002-9c49-fc8a4f56ebad None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:04.452172 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:04.452818 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:04.453133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:04.453409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:04.455212 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:04.467335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:04.468918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:04.478791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1417/2827] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:02:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:02:07.120448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=34 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:10.490593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04560fb-a9c0-452f-89bc-1be7ee6743c8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:10.491161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04560fb-a9c0-452f-89bc-1be7ee6743c8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:10.491161 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04560fb-a9c0-452f-89bc-1be7ee6743c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:10.508470 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a04560fb-a9c0-452f-89bc-1be7ee6743c8 None None] MFA Rules not processed for user `c89a6bacbd004d99837d8d092352f18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:10.510299 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a04560fb-a9c0-452f-89bc-1be7ee6743c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:10.564890 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1411/2828] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:02:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:10.794044 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:10.795602 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:10.807960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cf61-5207-4aab-bc99-22ad99d04dbd service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:10.811506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:10.812143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:10.812425 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:10.812701 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:10.814433 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:10.824327 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:10.825767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:10.833715 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1418/2829] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:02:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:02:20.560435 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=25 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:20.830165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=38 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:28.894743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b8869-8b4a-49d5-975c-d444ecf01e11 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:28.895281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b8869-8b4a-49d5-975c-d444ecf01e11 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:28.895388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b8869-8b4a-49d5-975c-d444ecf01e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:28.947240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4f5b8869-8b4a-49d5-975c-d444ecf01e11 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:28.948464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4f5b8869-8b4a-49d5-975c-d444ecf01e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:29.018751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1412/2830] 10.222.0.47 () {58 vars in 1009 bytes} [Sat Jul 16 13:02:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:29.231492 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:29.232877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:29.329550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473b69b-8dd9-4a64-a925-bf22b62aca18 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:29.334578 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:29.335127 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:29.335404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:29.335675 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:29.344277 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:29.345872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:29.353647 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1419/2831] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 13:02:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 13:02:39.015961 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=40 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:39.350599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=41 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:48.729439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:48.729879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:48.729997 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:48.749342 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] MFA Rules not processed for user `d1051f87dc384cfe86485b5e290982e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:48.750251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:48.757469 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1413/2832] 10.222.0.47 () {58 vars in 967 bytes} [Sat Jul 16 13:02:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:48.769661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:48.780186 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a071f-ef17-4ae8-bf0d-e178f4c9283d service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:48.785188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d7e37eb8b3e44d5a7f2be9af4a02c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:48.785819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:48.786104 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:48.786383 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:48.794962 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:48.796525 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:48.804730 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1420/2833] 10.222.0.47 () {60 vars in 1384 bytes} [Sat Jul 16 13:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 13:02:51.483818 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:51.504311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc34b-3f19-4835-b726-f3f695351745 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:51.504311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:51.504311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:51.504311 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:51.504778 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:51.505614 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:51.519240 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:51.520833 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:51.531968 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1414/2834] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:02:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:02:51.718998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:51.729589 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6257b-f514-4746-bee1-f772ad4ff41e service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:51.735278 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:51.739110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:51.739110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:51.739110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] PATH_INFO: `/v3/users/b68e5780b6324fc48b45b38503d2bcb6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:51.739110 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68e5780b6324fc48b45b38503d2bcb6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:51.739910 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:51.902466 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68e5780b6324fc48b45b38503d2bcb6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:51.909888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68e5780b6324fc48b45b38503d2bcb6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:51.923872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68e5780b6324fc48b45b38503d2bcb6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:51.924581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68e5780b6324fc48b45b38503d2bcb6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:51.974758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a23eea55-9d82-455f-8499-4d1243484a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68e5780b6324fc48b45b38503d2bcb6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:51.981777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1421/2835] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:02:51 2022] DELETE /identity/v3/users/b68e5780b6324fc48b45b38503d2bcb6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:02:52.241281 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:52.252879 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6cf60-3c80-4f10-a216-564941e2559d service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:52.313565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:52.314456 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.314893 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.315042 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] PATH_INFO: `/v3/projects/a5927ebc582d49538443e497cc15519d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.316256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5927ebc582d49538443e497cc15519d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:52.318403 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:52.437884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5927ebc582d49538443e497cc15519d', 'request_id': 'req-9004ea83-e9c4-47f0-827c-7e676c1d94be'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:52.443916 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5927ebc582d49538443e497cc15519d', 'request_id': 'req-9004ea83-e9c4-47f0-827c-7e676c1d94be'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:52.512801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5927ebc582d49538443e497cc15519d', 'request_id': 'req-9004ea83-e9c4-47f0-827c-7e676c1d94be'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:52.523856 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] The token cache is being invalidate because project a5927ebc582d49538443e497cc15519d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:02:52.524173 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:52.524800 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9004ea83-e9c4-47f0-827c-7e676c1d94be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:02:52.526353 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1415/2836] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:02:52 2022] DELETE /identity/v3/projects/a5927ebc582d49538443e497cc15519d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:02:52.531958 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.532257 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.532543 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.550740 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:52.551947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:52.612227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1422/2837] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:52.620015 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.620360 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.620684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.639551 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:52.640746 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:52.648873 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1416/2838] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:52.652964 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:52.663470 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e2c89-5ff7-4baf-939a-5893daf43382 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:52.668041 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:52.668905 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.669216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.669488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.670202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:02:52.670502 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:52.672182 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:52.683965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1423/2839] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:02:52.688483 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:52.697777 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60767880-0792-4acb-87fc-0e7481ae6181 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:52.702948 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:52.703591 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.703876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.704169 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.704846 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:52.706765 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:52.834255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1417/2840] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:02:52 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:02:52.838427 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:52.848513 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6676-2c54-4f4a-be06-475f7b27109c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:52.894568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:52.906957 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:52.907621 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:52.907900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:52.908173 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:52.908861 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:52.910819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:53.015777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1424/2841] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:02:52 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:02:53.019688 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:53.030085 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94fe25-454e-4eb4-acef-002982545437 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:53.034706 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:53.035418 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:53.035703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:53.035983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:53.036771 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:53.038411 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:53.049292 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1418/2842] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:02:53.053628 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:53.064083 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a23b2-79ac-4a0c-98c4-46202c6700b5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:53.068834 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:53.070822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:53.070822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:53.070822 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:53.071137 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:53.072422 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ef7f4-3773-4bfe-808c-120bf8f99feb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:53.084360 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1425/2843] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 13:02:53.088492 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:53.100004 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7adaaf-dba5-407b-ad58-77b943e69015 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:53.104696 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:53.105685 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:53.106556 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:53.106836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] PATH_INFO: `/v3/projects/49b939813cbc4a9e96a046e1e73be7dc/users/090dc02eca794885aebd4f246c185d9f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:53.124153 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b939813cbc4a9e96a046e1e73be7dc, user_id=090dc02eca794885aebd4f246c185d9f, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:53.130933 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:53.197300 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1419/2844] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:02:53 2022] PUT /identity/v3/projects/49b939813cbc4a9e96a046e1e73be7dc/users/090dc02eca794885aebd4f246c185d9f/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:02:53.208329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:53.208615 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:53.208888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:53.235103 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] MFA Rules not processed for user `090dc02eca794885aebd4f246c185d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:02:53.236295 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:53.300141 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1426/2845] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:02:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:02:53.312738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:02:53.314593 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:02:53.329667 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:02:53.399219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485f80-63d8-4d77-b73a-2e5cf69ae2b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:02:53.402931 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff6db7c70f64a649a64313389f7e9bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:02:53.403577 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:02:53.403855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:02:53.404132 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:02:53.412607 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:02:53.415884 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:02:53.423270 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1420/2846] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:02:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:03:03.297225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:03.557705 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=32 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:42.165420 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:42.166936 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:42.233902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ac8b-aa59-48b7-8445-e8f193bf7009 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:42.238495 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:42.239379 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:42.239666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:42.239940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] PATH_INFO: `/v3/users/c89a6bacbd004d99837d8d092352f18f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:42.241143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89a6bacbd004d99837d8d092352f18f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:42.243876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:42.400418 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89a6bacbd004d99837d8d092352f18f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.408635 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89a6bacbd004d99837d8d092352f18f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.421283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89a6bacbd004d99837d8d092352f18f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.421979 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89a6bacbd004d99837d8d092352f18f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.499217 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d196fd1c-dbd1-4f2e-a5af-ec47403cd75c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89a6bacbd004d99837d8d092352f18f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.506026 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1427/2847] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:03:42 2022] DELETE /identity/v3/users/c89a6bacbd004d99837d8d092352f18f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:42.735509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:42.746937 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d5c14-9e9e-4643-8378-9ec4132a151b service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:42.806382 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:42.807302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:42.807584 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:42.807862 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] PATH_INFO: `/v3/projects/ada92e33c07f4be7b2d752550c381dfb` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:42.809069 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada92e33c07f4be7b2d752550c381dfb)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:42.811011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:42.922737 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada92e33c07f4be7b2d752550c381dfb', 'request_id': 'req-d400db16-b2c2-406d-b67b-ab833c35be79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.929118 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada92e33c07f4be7b2d752550c381dfb', 'request_id': 'req-d400db16-b2c2-406d-b67b-ab833c35be79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:42.990817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada92e33c07f4be7b2d752550c381dfb', 'request_id': 'req-d400db16-b2c2-406d-b67b-ab833c35be79'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:43.002307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] The token cache is being invalidate because project ada92e33c07f4be7b2d752550c381dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:03:43.002589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:43.003206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d400db16-b2c2-406d-b67b-ab833c35be79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:43.004736 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1421/2848] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:03:42 2022] DELETE /identity/v3/projects/ada92e33c07f4be7b2d752550c381dfb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:03:43.009780 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.010068 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.010340 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.027987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:43.029211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:43.087643 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1428/2849] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 13:03:43.094888 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.095179 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.095448 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.114652 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:43.115849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:43.123723 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1422/2850] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:03:43.128811 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.138403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d111b0-4e63-4469-9872-245499ab52f1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.143162 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.143800 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.144082 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.144355 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.145105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:03:43.145416 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.147004 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.158618 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1429/2851] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:03:43.162681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.174550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994b63-67a1-40d8-8287-a9c12084a750 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.179841 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.180182 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.180468 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.180743 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.181482 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.183358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.267374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1423/2852] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:03:43 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 13:03:43.271661 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.283051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec789d-4015-41ba-8d22-b7e7e3978921 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.286680 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:43.338662 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.339326 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.339613 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.339889 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.340568 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.342501 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.408888 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1430/2853] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:03:43 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 13:03:43.413321 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.424912 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a915-9fff-40cd-8d61-73f12d38fc91 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.429786 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.436294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.436294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.436294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.436294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.436294 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.443891 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1424/2854] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:03:43.447752 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.457926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fbba6-923b-4a90-bcd1-fd98b1bb66c2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.462579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.463225 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.463579 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.463833 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.464550 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.466177 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ab9fc-05e4-48bb-827e-563466d0a31b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.477184 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1431/2855] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:03:43.481193 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.497233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315599b2-b6e1-4c0e-8468-26a14bdd5b3b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.497233 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.497457 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.497731 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.498011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] PATH_INFO: `/v3/projects/57b0103008c445a5b4b980992a61f4c8/users/f9dc944192d24af0b1e2f64b7e071a3b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.512428 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b0103008c445a5b4b980992a61f4c8, user_id=f9dc944192d24af0b1e2f64b7e071a3b, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.515108 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.602932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1425/2856] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:03:43 2022] PUT /identity/v3/projects/57b0103008c445a5b4b980992a61f4c8/users/f9dc944192d24af0b1e2f64b7e071a3b/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:03:43.608084 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.608404 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.608689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.628384 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:43.629624 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:43.689316 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1432/2857] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:03:43.699287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:43.701081 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:43.732900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:43.759389 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc99354-1c5b-4887-90b6-4279f7e39bcc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:43.763338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:43.764032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:43.764325 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:43.764589 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:43.774438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:43.775848 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2002f2-4df1-4bd9-88c9-c65f09a7d280 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:43.786526 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1426/2858] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:03:47.207987 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:47.209841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:47.227872 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf4625-cd35-4b16-9b39-fb81da65cfa0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:47.234038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:47.237309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:47.237785 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:47.238224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] PATH_INFO: `/v3/users/090dc02eca794885aebd4f246c185d9f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:47.239705 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=090dc02eca794885aebd4f246c185d9f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:47.241896 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:47.459091 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090dc02eca794885aebd4f246c185d9f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:47.475786 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090dc02eca794885aebd4f246c185d9f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:47.492758 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090dc02eca794885aebd4f246c185d9f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:47.493700 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090dc02eca794885aebd4f246c185d9f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:47.574165 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090dc02eca794885aebd4f246c185d9f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:47.576150 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:47.582599 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1433/2859] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:03:47 2022] DELETE /identity/v3/users/090dc02eca794885aebd4f246c185d9f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:47.587022 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:47.587527 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:47.587642 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:47.607099 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:47.608450 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:47.678451 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1427/2860] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:03:47.687876 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:47.689443 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:47.755094 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b7fc8-4583-42a5-9c87-d598831353ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:47.759208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:47.759523 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:47.759809 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:47.760087 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:47.771854 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:47.773816 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256d1c-0dfb-402a-b119-6d0cfa3b4ba7 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:47.784415 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1434/2861] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:03:48.214934 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:48.216367 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:48.231985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15e7b5-fba0-42de-af9c-0927a2465396 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:48.239130 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:48.245429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.245429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.245429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] PATH_INFO: `/v3/projects/49b939813cbc4a9e96a046e1e73be7dc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.245429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b939813cbc4a9e96a046e1e73be7dc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:48.245429 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:48.379568 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b939813cbc4a9e96a046e1e73be7dc', 'request_id': 'req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:48.388359 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b939813cbc4a9e96a046e1e73be7dc', 'request_id': 'req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:48.494219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b939813cbc4a9e96a046e1e73be7dc', 'request_id': 'req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:48.510100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] The token cache is being invalidate because project 49b939813cbc4a9e96a046e1e73be7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:03:48.512801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:48.512801 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9561c5d4-fe0e-48dc-9824-f3cef7b5da97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:03:48.513392 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1428/2862] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:03:48 2022] DELETE /identity/v3/projects/49b939813cbc4a9e96a046e1e73be7dc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:48.518251 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.519532 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.520062 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.540497 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:48.542011 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:48.623455 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1435/2863] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:03:48.626295 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.626769 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.626860 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.652439 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:48.654089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:48.666135 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1429/2864] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 13:03:48.670603 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:48.687924 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdfab6-29d5-48e3-a706-9992eda18704 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:48.721593 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:48.722275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.722971 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.723388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.724275 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 13:03:48.724747 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:48.726782 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:48.745537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1436/2865] 10.222.0.47 () {58 vars in 1152 bytes} [Sat Jul 16 13:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 13:03:48.746822 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:48.761480 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1779761-2301-4e30-9fd6-3dbedee72a12 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:48.771602 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:48.775086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.775086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.775086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.775086 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:48.776638 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:48.917388 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1430/2866] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:03:48 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:03:48.922651 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:48.933503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8266f-731c-4adf-bdfb-388aefae355f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:48.997562 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:48.998453 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:48.998732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:48.999014 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:48.999716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.001913 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.088717 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1437/2867] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:03:48 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:03:49.092698 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.105703 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31846121-9a0b-419d-9017-cd0fc1e15cf7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.111536 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.112059 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.112346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.112627 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.113459 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.115129 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.131975 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1431/2868] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:03:49.132849 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.144814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbbe7-82b5-4e96-bdf7-d8c7b349f0aa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.151192 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.151960 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.152161 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.152437 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.153197 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.154832 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a215801-dd38-47c7-8286-4b0ebc4f4d9b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.173660 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1438/2869] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 13:03:49.181138 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.192330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8817-ffcf-49b7-a634-5b0a4ffbf4d5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.198714 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.199784 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.200171 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.200517 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] PATH_INFO: `/v3/projects/6c2a8f870b834819992f1a8fd0955fcd/users/c56cf1e8cb644a5c977a0d8631214c44/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.228312 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:49.237174 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2a8f870b834819992f1a8fd0955fcd, user_id=c56cf1e8cb644a5c977a0d8631214c44, role_id=cebbfbe0d4fa4c5881ee638fb0a2cdf9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.243538 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fc355-5bb0-44f9-aadc-04742b6a6a04 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.318440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1432/2870] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:03:49 2022] PUT /identity/v3/projects/6c2a8f870b834819992f1a8fd0955fcd/users/c56cf1e8cb644a5c977a0d8631214c44/roles/cebbfbe0d4fa4c5881ee638fb0a2cdf9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:49.330933 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.331224 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.331506 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.349926 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:49.374407 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] MFA Rules not processed for user `c56cf1e8cb644a5c977a0d8631214c44`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:49.376179 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:49.418686 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.418976 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.419260 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.450985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] MFA Rules not processed for user `f9dc944192d24af0b1e2f64b7e071a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:49.450985 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:49.459105 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1439/2871] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:03:49.469814 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.472121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:49.526419 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1433/2872] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 13:03:49.530866 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.532998 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:49.544434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b1a28-980e-43a0-b07c-8d0066686af2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.548488 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.549169 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.549491 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.549775 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.559118 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.560676 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.571164 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1440/2873] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:03:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 13:03:49.606474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ded39-659b-461f-bd5b-42d6a993cfa0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.611533 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.612194 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.612472 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.612747 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.613488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.615374 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.730126 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1434/2874] 10.222.0.47 () {60 vars in 1152 bytes} [Sat Jul 16 13:03:49 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 13:03:49.733737 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.744881 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a747b-68e8-473a-835d-b94b19c70567 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.809656 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.810335 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.810617 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.810891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.811611 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.814712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.871050 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1441/2875] 10.222.0.47 () {60 vars in 1143 bytes} [Sat Jul 16 13:03:49 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 13:03:49.876587 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.887561 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f360c9-4b7e-4532-a3ed-4a8f8d1a9bff admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.892402 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.893148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.893437 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.893717 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.894453 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.896054 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:49.907366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1435/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:03:49.911698 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:49.927984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf009136-5e22-4c09-b123-5c56ac580d08 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:49.932890 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:49.933877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:49.934133 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:49.934398 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] PATH_INFO: `/v3/projects/507de827011343ffadda823d1e275789/users/55dfb3784b7b4b9ca0d7dd1bb771078b/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:49.950537 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=507de827011343ffadda823d1e275789, user_id=55dfb3784b7b4b9ca0d7dd1bb771078b, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:49.953388 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:50.025795 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1442/2877] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:03:49 2022] PUT /identity/v3/projects/507de827011343ffadda823d1e275789/users/55dfb3784b7b4b9ca0d7dd1bb771078b/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:50.029779 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:50.041546 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b6351-6de0-4fcb-9a4c-d373b95502fc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:50.092348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:50.102317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:50.102983 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:50.103263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:50.103632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:50.104306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:50.105965 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d37909-1821-46e1-9761-0d4704fe64c6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:50.117162 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1436/2878] 10.222.0.47 () {58 vars in 1121 bytes} [Sat Jul 16 13:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 13:03:50.121273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:50.128468 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:03:50.132679 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42a1b6-7b92-4746-902d-2ef37bb7ce16 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:50.137776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:50.138674 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:50.138961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:50.139237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] PATH_INFO: `/v3/projects/507de827011343ffadda823d1e275789/users/55dfb3784b7b4b9ca0d7dd1bb771078b/roles/a495d38f67e447309647ac782785bd87` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:50.141413 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=507de827011343ffadda823d1e275789, user_id=55dfb3784b7b4b9ca0d7dd1bb771078b, role_id=a495d38f67e447309647ac782785bd87)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:50.144143 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:50.154862 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1443/2879] 10.222.0.47 () {60 vars in 1482 bytes} [Sat Jul 16 13:03:50 2022] PUT /identity/v3/projects/507de827011343ffadda823d1e275789/users/55dfb3784b7b4b9ca0d7dd1bb771078b/roles/a495d38f67e447309647ac782785bd87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:03:55.856355 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:55.856935 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:55.857070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:55.885859 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] MFA Rules not processed for user `55dfb3784b7b4b9ca0d7dd1bb771078b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:03:55.887123 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:55.964802 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1437/2880] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:03:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 13:03:55.986908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:03:55.990086 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:03:56.069794 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4e7a7-8cdd-473d-82c0-022a29f9f787 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:03:56.075763 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:03:56.076409 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:03:56.076689 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:03:56.076961 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:03:56.134273 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:03:56.135866 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c80aa-d93f-439d-aa5e-6e6119f15270 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:03:56.145638 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1444/2881] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:03:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 13:04:05.961307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=39 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:06.141444 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:15.548630 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:15.610832 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e3ac8-6551-4d8e-b74b-4ef208ae3291 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:15.615471 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:15.616335 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:15.616621 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:15.616897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] PATH_INFO: `/v3/users/c56cf1e8cb644a5c977a0d8631214c44` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:15.618185 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56cf1e8cb644a5c977a0d8631214c44)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:15.620097 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:15.769550 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56cf1e8cb644a5c977a0d8631214c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:15.781358 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56cf1e8cb644a5c977a0d8631214c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:15.783065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56cf1e8cb644a5c977a0d8631214c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:15.783770 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56cf1e8cb644a5c977a0d8631214c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:15.790496 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56cf1e8cb644a5c977a0d8631214c44'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:15.858481 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1438/2882] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:04:15 2022] DELETE /identity/v3/users/c56cf1e8cb644a5c977a0d8631214c44 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:04:15.864020 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:15.864500 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:15.864614 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:15.881190 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] MFA Rules not processed for user `29ed4c3c304b4678a24977fbdcf04135`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:04:15.885789 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:15.939595 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1445/2883] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:04:15.949306 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:15.950708 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:16.007032 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b054ba7-f501-4aab-bcd5-46621178c1d7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:16.010918 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:16.011559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:16.011836 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:16.012106 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:16.021491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:16.022897 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f9e2c-49d6-4aa7-86e6-e86303222e0a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:16.032905 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1439/2884] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:04:16.441237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:16.441237 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:16.454348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dd2ab-8c95-453c-b70d-ffc4fb8bbf41 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:16.459847 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:16.460716 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:16.460998 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:16.461301 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] PATH_INFO: `/v3/projects/6c2a8f870b834819992f1a8fd0955fcd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:16.462547 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2a8f870b834819992f1a8fd0955fcd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:16.464447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:16.555392 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2a8f870b834819992f1a8fd0955fcd', 'request_id': 'req-3dae09e3-7697-4801-9015-3c5a59b73dc9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:16.561828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2a8f870b834819992f1a8fd0955fcd', 'request_id': 'req-3dae09e3-7697-4801-9015-3c5a59b73dc9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:16.658762 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2a8f870b834819992f1a8fd0955fcd', 'request_id': 'req-3dae09e3-7697-4801-9015-3c5a59b73dc9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:16.669633 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] The token cache is being invalidate because project 6c2a8f870b834819992f1a8fd0955fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:04:16.669915 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:16.670531 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:04:16.672059 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1446/2885] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:04:16 2022] DELETE /identity/v3/projects/6c2a8f870b834819992f1a8fd0955fcd => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:04:24.925012 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:24.925814 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:24.925928 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:24.943460 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] MFA Rules not processed for user `f9dc944192d24af0b1e2f64b7e071a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:04:24.944758 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:24.961148 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:24.998487 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1440/2886] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:04:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:04:25.035997 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:25.037434 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:25.096908 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae09e3-7697-4801-9015-3c5a59b73dc9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:25.100541 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:25.101211 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:25.101494 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:25.101767 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:25.103575 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:25.142109 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:25.174732 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:25.176477 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:25.189450 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1447/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:04:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:04:26.155843 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:26.157876 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:26.219954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bded4-5b22-4729-8574-78e19a2e0e71 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:26.225302 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:26.225970 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:26.226287 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:26.226515 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:26.228766 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:26.247100 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:26.248662 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:26.273066 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1441/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:04:35.241202 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:36.256632 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=41 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:04:46.201030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:04:46.202478 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:46.214432 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ffae-2cc2-477e-abb2-44dc48696732 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:04:46.217969 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:04:46.218625 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:04:46.219448 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:04:46.219721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:04:46.221459 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:04:46.281994 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:04:46.283447 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:04:46.292937 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1448/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:04:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:04:56.288596 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=30 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:06:02.556256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:06:02.592569 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bd431-4a69-44c4-a298-a5be6b3e673c service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:06:02.596811 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:06:02.600817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:06:02.600817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:06:02.600817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:06:02.600817 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:06:02.631594 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:06:02.633070 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:06:02.641113 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1442/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:06:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:06:12.637522 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=27 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:07:04.797536 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:07:04.799079 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:04.942607 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7800f9-230f-4ae4-987d-74701f6c3ab3 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:07:04.948941 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:07:04.949734 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:04.950203 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:04.950499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:04.953208 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:04.969431 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:07:04.971181 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ddfad-2031-4265-bb25-27b43a96d96f service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:07:04.981748 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1449/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:07:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:07:05.164206 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:07:05.166029 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:05.195574 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8dfff-3d4d-44a5-8cf4-be82d1e42369 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:07:05.202049 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:07:05.202699 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:05.203443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:05.203443 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:05.204987 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:05.218901 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:07:05.223954 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:07:05.268338 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1443/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:07:07.223329 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:07.223791 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:07.223900 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:07.240561 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] MFA Rules not processed for user `f9dc944192d24af0b1e2f64b7e071a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:07:07.241779 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:07.248545 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1450/2893] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:07:10.483858 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:07:10.495763 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147a9d-f80e-4afd-a381-319bfced1e21 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:07:10.500759 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:07:10.501430 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:10.501715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:10.501995 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:10.504000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:10.516120 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:07:10.518255 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:07:10.528509 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1444/2894] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:07:17.235918 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=62 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:07:20.524440 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=12 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:07:29.659191 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:07:29.673185 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65299118-caf5-418d-9657-134b53f13bf7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:07:29.679567 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:07:29.680196 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:29.680475 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:29.680746 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:29.682659 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:29.694518 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:07:29.699841 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:07:29.706512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1451/2895] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:07:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 13:07:34.824588 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:07:34.826013 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:34.841180 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b274126-7241-4e4f-bfdc-6bd80880b808 service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:07:34.846622 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0faa6a54416f420bbdec92832d404efd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:07:34.847184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:07:34.847463 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:07:34.847738 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:07:34.850544 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:07:34.862755 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:07:34.864317 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a67900-6a3c-4cf6-b66d-f5fda39f7971 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:07:34.876366 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1445/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:07:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:07:39.702123 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:07:44.872249 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:09:23.684511 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:09:23.686113 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:09:23.719319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cfe69-4611-4027-93fd-d9be9bf7d70e service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:09:23.723309 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:09:23.724030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:23.724354 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:23.724687 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:23.726574 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:09:23.737974 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:09:23.739375 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:09:23.748271 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1452/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:09:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:09:33.744074 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=15 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:09:35.322503 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:35.323018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:35.323018 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:35.341296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] MFA Rules not processed for user `f9dc944192d24af0b1e2f64b7e071a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 13:09:35.342514 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:09:35.349996 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1446/2898] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jul 16 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 13:09:35.386144 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:09:35.395666 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d72d8-64e4-4ef9-9829-5c466735a74f service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:09:35.399399 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10255724102414ca184fac122fd1e09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:09:35.400031 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:35.400310 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:35.400581 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:35.417563 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:09:35.419260 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:09:35.440897 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1453/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:09:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:09:35.725751 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:09:35.735869 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a055-bc0d-4921-913a-ee8ea0666289 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:09:35.739431 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:09:35.740065 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:35.740346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:35.740619 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:35.742375 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:09:35.751900 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:09:35.753348 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:09:35.761263 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1447/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:09:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:09:36.458285 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:09:36.467766 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e60ee-6a4f-465d-8c29-a3749bfb6e82 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:09:36.471756 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:09:36.472306 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:36.472590 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:36.472891 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:36.483714 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:09:36.485790 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:09:36.497403 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1454/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:09:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:09:45.757684 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=9 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:09:46.492819 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=19 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:09:55.527804 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:09:55.536491 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3c28-1eb3-4995-b4fd-69a0f1712a7d service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:09:55.540024 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:09:55.540656 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:09:55.540939 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:09:55.541250 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:09:55.550932 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:09:55.552333 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:09:55.561644 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1448/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:09:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:10:05.557559 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:11:09.553906 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:11:09.562512 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f52a4-b9b7-460e-8fc2-15556d29fadc service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:11:09.566255 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:11:09.567654 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:11:09.567940 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:11:09.568216 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:11:09.575777 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:11:09.577240 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:11:09.585835 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1455/2903] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:11:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:11:19.581559 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:12:02.470454 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:02.479601 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf923c-9e7a-4dc8-8e02-21981a1291f7 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:02.483343 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '246c8e2b8aec415496a87067875975a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:02.483982 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:02.484266 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:02.484545 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:02.486320 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:02.569659 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:02.571710 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:02.590474 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1449/2904] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:12:05.883984 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:05.896850 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc183-fc29-4a82-b68d-d6c0a170a2ec service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:05.902712 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:05.903353 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:05.903776 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:05.904051 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:05.905824 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:05.918215 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:05.920107 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:05.930526 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1456/2905] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:12:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:12:09.902745 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:09.904201 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:09.917967 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866acfac-8336-462c-ad87-e8844f20461d service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:09.922646 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:09.926715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:09.926715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:09.926715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] PATH_INFO: `/v3/users/f9dc944192d24af0b1e2f64b7e071a3b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:09.926715 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9dc944192d24af0b1e2f64b7e071a3b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:09.927256 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:10.128412 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9dc944192d24af0b1e2f64b7e071a3b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.140565 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9dc944192d24af0b1e2f64b7e071a3b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.141251 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9dc944192d24af0b1e2f64b7e071a3b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.141849 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9dc944192d24af0b1e2f64b7e071a3b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.150102 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9dc944192d24af0b1e2f64b7e071a3b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.235030 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1450/2906] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:12:09 2022] DELETE /identity/v3/users/f9dc944192d24af0b1e2f64b7e071a3b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:12:10.493725 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:10.504462 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5c5fc-f2a5-451e-95c1-535b5fabc5cf service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:10.569359 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:10.570503 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:10.570721 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:10.570965 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] PATH_INFO: `/v3/projects/57b0103008c445a5b4b980992a61f4c8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:10.572121 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b0103008c445a5b4b980992a61f4c8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:10.574038 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:10.677481 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b0103008c445a5b4b980992a61f4c8', 'request_id': 'req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.683764 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b0103008c445a5b4b980992a61f4c8', 'request_id': 'req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.765951 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b0103008c445a5b4b980992a61f4c8', 'request_id': 'req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.777066 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] The token cache is being invalidate because project 57b0103008c445a5b4b980992a61f4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:12:10.777381 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.778030 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:10.779670 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1457/2907] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:12:10 2022] DELETE /identity/v3/projects/57b0103008c445a5b4b980992a61f4c8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:12:10.783675 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:10.785146 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:10.799089 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b197d87-32f1-4da8-beff-669d040fba9c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:10.832296 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:12:10.855219 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:10.856131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:10.856407 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:10.856682 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] PATH_INFO: `/v3/users/55dfb3784b7b4b9ca0d7dd1bb771078b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:10.857883 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dfb3784b7b4b9ca0d7dd1bb771078b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:10.859808 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:11.077131 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dfb3784b7b4b9ca0d7dd1bb771078b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.089184 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dfb3784b7b4b9ca0d7dd1bb771078b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.089855 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dfb3784b7b4b9ca0d7dd1bb771078b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.090464 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dfb3784b7b4b9ca0d7dd1bb771078b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.097330 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dfb3784b7b4b9ca0d7dd1bb771078b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.186438 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1451/2908] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jul 16 13:12:10 2022] DELETE /identity/v3/users/55dfb3784b7b4b9ca0d7dd1bb771078b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 13:12:11.422200 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:11.424627 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:11.437188 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373d4c3-dfd7-4f69-bacb-1793ab3619ce admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:11.495319 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29ed4c3c304b4678a24977fbdcf04135', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d986eebab5a45d29f4ae51c867a19af', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:11.496111 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:11.496393 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:11.496669 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] PATH_INFO: `/v3/projects/507de827011343ffadda823d1e275789` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:11.497877 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=507de827011343ffadda823d1e275789)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:11.499774 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:11.520483 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:12:11.608991 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '507de827011343ffadda823d1e275789', 'request_id': 'req-a0c8e184-02ff-499c-b3bc-c96447c66c7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.615283 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '507de827011343ffadda823d1e275789', 'request_id': 'req-a0c8e184-02ff-499c-b3bc-c96447c66c7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.699948 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '507de827011343ffadda823d1e275789', 'request_id': 'req-a0c8e184-02ff-499c-b3bc-c96447c66c7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.710449 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] The token cache is being invalidate because project 507de827011343ffadda823d1e275789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 13:12:11.710741 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.711390 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 13:12:11.712919 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1458/2909] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jul 16 13:12:11 2022] DELETE /identity/v3/projects/507de827011343ffadda823d1e275789 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 13:12:11.786167 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:12:11.787628 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:11.868780 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c6536-1766-4a04-be50-20e6657353ee admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:12:11.932346 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:12:11.940011 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:12:11.940681 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:12:11.941000 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:12:11.941307 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:12:11.943237 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:12:11.955776 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:12:11.957498 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed280015-8177-4832-bddf-fa4b13441ead service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:12:11.969595 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1452/2910] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:12:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:12:21.928781 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:12:22.248488 dv-cinder-836973-12-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 13:17:16.747902 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 13:17:16.749365 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 13:17:16.834250 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8e184-02ff-499c-b3bc-c96447c66c7b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 13:17:16.839305 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c73b6a617d4546599d336eeb5b45e858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0ab5797a37f470ebec8ac33b02a8e60', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 13:17:16.839947 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 13:17:16.840227 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 13:17:16.840499 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 13:17:16.849264 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 13:17:16.850828 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252762c-76f5-456f-bde3-02e64de7ea49 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 13:17:16.860609 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1459/2911] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jul 16 13:17:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 13:17:26.856348 dv-cinder-836973-12-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=37 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}